Empirical studies of ConDiS Conducting

My composition “Kuuki no Sukima” for nine instruments is specially written for use of my artistic research project ConDiS – Conducting Digital System. There the conductor wears a digital/ analog glove that can control the overall volume of the outcoming electronic sound, the overall tempo of the performance and synchronization of the live instruments and the computer.

Original idea was to have the conductors role a bit larger by allowing him to control electronic sonority of individual instrumental groups as well as give him control of other features such as panning of the sound (move the sound in space) and the sound timber (sound color) by opening and closing spectral filters, changing reverb and delay time and envelopes etc.

All these things are available, the conductor can control individual instrumental groups by using signs of one, two, three, four and five fingers (see Notation and control from March 13, 2017.

Comparison of performances

The following is a comparison of the conductor’s use of ConDiS between performances in Harpa, Reykjavik and Schæffergården, Copenhagen.

Example 1.

Instructions from measure three of the opening of the 1st movement.

Conducting at Schæffergården.

Comparing the opening of the 1st. movement shows that the conductor has an identical way to conduct. Not to surprise since the use of ConDiS is written in the score.

Conducting at Harpa.

Example 2.

Now compare:

Conducting at Schæffergården.

Conducting at Harpa.

Although not written in the score the conductor in this example activates the volume value control at a very similar point. At the Harpa concert, the conductor activates the volume value control a bit earlier then decides to wait till after the Flute has ended. Then similarly to the Schæffergården she activates it again just before the Horn but deactivates it sooner in Harpa. It should be noticed that the Harpa concert was the first of four concerts and Schæffergården the last one. For me, it is obvious that the conductor has become much more comfortable with the ConDiS system.

Example 3.

Again the conductor follows instructions written in the score.

Conducting at Schæffergården.

Conducting at Harpa.

Example 4.

And the closing measures of 1st. movement. The conductor follows written instructions identically.

Volume value Control – 3rd. movement

Continuing to analyze the conductor’s volume controlling gestures using the ConDiS conducting digital system. Now looking at 3rd. movement of Kuuki no Sukima. The first instance of volume value controlling happens around measure 29. As can be seen in this example from the score there are no written instructions, therefore, this is totally the conductor’s decision to control the electronic sound.

Example 1.

From the video clip below the conductor adds electronics at the end of the piano phrase or where the horn makes increasing air sound. The conductor is here consciously conducting the electronics and the acoustics the same way as she would be conducting transformation of one instrumental sound to another. It is important that the conductor has now become so familiar with the ConDiS system that she is now using that tool as she does when conducting the performers.

Example 2.

The following example shows measure 41-44 although the video clip starts about a measure earlier and ends a measure later. The main reason being a page turn that was not easy to copy an paste into such a small frame. In the missing measure before the conductor has to press the 3rd button to stop the DAW and then in measure 41 press the 4th button to have the DAW to jump (if it were off) and synchronize with the score at the beginning of that measure.  At the same time, the conductor decides to activate the volume value control which is not a bad idea but let’s take a look at how it did go.

The following video example shows the conductor making a slight mistake but corrects it on the fly so to speak. There she has to stop the DAW and jump to next marker (buttons 3 and 4), she decides to close her hand for volume control but realizes that it was a bit too early, corrects it, does the button 3 and 4 and then she activates the volume control. Important to notice the flow of the correction how “naturally” she opens her hand again, presses the buttons and then activates the volume control.

Example 3.

The following example showing measures 54-63 is a very good example of the use of the ConDiS system, as the conductor needs both to stop the DAW and synchronize and then set a new tempo (metronome 92). Immediately she then activates a volume value control to reduce electronic sounds, although not written in the score.

The accompanying video with the same measures as above shows clearly how the conductor reduces the volume value of the electronics, probably because she has found the echo effects following the Pizzicato (plucked strings) to be too strong. Soon, she then decides to increase the strength so that the echo comes in. This is still a very good example of how the conductor himself determines the balance between the sound and electronics. It was my intention to keep the echo effect strong at the beginning of this pizzicato section, but I have to confess that the conductor’s interpretation is very good and gives the music a lot more life. There again comes to the conductor’s role in the music movement. Even if generally accepted that the conductor’s job is to put themselves into the mind of the composer and interpret music from it, then it is inevitable that the conductor sets his mark on the performance. The fact that the conductor puts his character in the composition is for me, as a composer, very important. That way the music comes to life, from being black dots on the manuscript paper to a sea of sounds, tones, and chords.

Example 4.

The following example from measure 82-87 again shows the conductor decides to activate the volume value control and add a bit more electronics to the performance. This section is very soft very subtle and therefore since all the electronic sounds are directly connected to the sounds of the instruments, so are the electronics. Still, we can hear that there is a very subtle change of tone color as the conductor raises her arm. A very beautiful movement in the piece and for me shows clearly how the ConDiS system can be used to make little nuances that are so important for my music.

The following video shows the performance of the above score. Notice how the conductor decides to add more electronics to the mix and how ads a slight color change to the harmonic spectrum.

Example 5.

Another example of how the conductor decides or makes a musical decision to take over the volume control and lower the electronics just before the piano enters. The conductor definitely feels that the electronic level need to be decreased so that the piano entrance is more clear.

Volume value Control – 2nd. movement

In my last blog, I went through analyses of the conductor’s volume value controlling gestures. Now, let’s take a closer look at the second movement where there is more complexity in musical action and therefore less time for the conductor to adjusting the electronic volume.

Example 1.

After a fermata and a jump forward (button 3 and 4) there is a left-hand pizzicato in the Double Bass a high note in the Violin and then a short Flute motive. Shown with red arrows in the illustration below.

Although, I personally would have liked to have a bit more volume on the Double Bass electronics I must confess that the conductor does a quite interesting job by not increasing the volume until the last sustained note of the flute. Let’s take a look at the performance.

Listen how the sustained note of the Flute gets an extra space when the conductor decides to add more volume value to that note. Keep in mind that this is not written in the score, here it is a musical judgment of the conductor to do so. This is very much to my liking and proofs for me the artistic value of the ConDiS system. It comes natural, it ads natural feeling to the flow, it is spontaneous and therefore an extension of the interaction between the acoustic and the electronic sounds. This could never happen except with the ConDiS system, which gives the conductor the ability to shape the music through her musical experience and knowledge. It gives the conductor a tool to express her musicality to the whole performance, not only the instrumental part.

Example 2.

The next example is from measures 16 – 21 where the left-hand pizzicato motive from measure 11 is repeated. Again nothing is written in the score for the conductor to adjust the electronic volume value. It is, therefore, her musical judgment to add more electronics to the pizzicato. By doing so, the conductor brings out slightly variated repetition causing a very interesting effect, making the repetition much more effective.

Conductor ads electronic volume to the pizzicato motive of the Double Bass.

Example 3.

In example 3 there is a written instruction to the conductor to raise the volume. It should probably be to increase the volume by raising her arm. there is also an OK sign there that is probably a leftover from earlier versions when the conductor could also activate panning and effect control with thumb up sign. Control that I decided to leave out firstly since it was not part of the traditional conducting job and secondly since it was too much task for the conductor. I decided to leave the OK sign there for the conductor to make absolutely sure that the volume value control would be activated and also just to find out her reaction.

Volume value Control

What I want to focus on in this blog is the conductor’s use of her arm to control the overall volume of the electronics. As I did show in earlier blogs she does at the beginning adjust the overall volume but does she adjust it again?

Kuuki no Sukima 1st. movement

The answer is Yes she does and by looking at performance in Copenhagen we find out that she does it seven times during the 107 measures of the first movement. I choose to use the Copenhagen concert since it was the last concert of the four during the Nordic Tour. That means the conductor had got more acquainted and experienced with ConDiS and was using the glove’s possibilities in a more relaxed, freer and natural way than at the beginning.

Four of these seven volume controls are written in the score. Why? They are written in the score as part of my research to find out how the conductor responds if volume control values are written in the score. It should be stated that I did not ask the conductor specifically to make changes there but told her that these could be interesting spots in the piece to adjust the balance of electronics and live instruments. I wrote them there yes as said before because these were good places to adjust or even play a little with volume value changes. Perhaps I wrote them there so the use of the ConDiS could be a bit more noticeable for the audience. Perhaps I was being a little bit show-offy. Let’s look at these places in the first movement and see and hear how they work.

The first volume control is written in the score at measure 3 and 4, see illustration below. It is well illustrated in my former blogs and does therefore not need more explanation.

Example 1.

Here is a video clip from the same spot of measure

Example 2.

The Bass Clarinet comes in with a combination of irregular key slap and trill and soon after the Horn plays a note that has a crescendo from p (soft) to f (loud). At the same time, the conductor decides to give the Horn a bit more electronic support as can be heard in the below video clip.

The electronic effect can easily be heard as the conductor raises her hand and therefore it is clear that she is affecting the overall value of the electronic sound. Since the conductor opens her hand in the topmost position of the arm the electronic value stays at a high value until she does the next adjustment which occurs only a measure or two later (approx. measure 24) as can be seen in the next example.

Example 3.

Video of the same measures.

It is clear that the conductor decides to lower the electronic sound value when she hears that the strings are getting a little bit strong and then after the clarinet comes in decides to give it a bit more electronic sound value by raising her arm up to almost top position but then decides to lower it a little bit and deactivates the volume control around 75% of the total value or with the arm just above the middle level. It has to be pointed out that the position of the arm and volume value can vary since it is all based on the relative position meaning that the arm position value is based on the last arm position value. This means that if the conductor deactivated with 75% volume value with her arm about 2/3 up from the lowest position she will keep that value as a starting point next time she activates the volume value control no matter in what position her arm is at that moment. This had to be done since otherwise there would be an audible jump in the volume when she activates the volume value control since the arm is most likely not going to be in the same position as the last time she deactivated the control. There is more written about this and other technical issues in the technological chapter of this essay.

Similar things happen in the next example from measure 63 – 68.

Example 4.

Video of the same measures:

Back to score writing and sonic analyzes

This last week I have been proofreading through the final version of Kuuki no Sukima. As recommended by conductor Halldis Rønning I did write two versions of the score, one with all the electronic graphics and another without except audio graphics indicating the overall volume. The score with all the electronic graphics is named Composer-Conductor and the other one Performance Score. Instead of adding all instructions to the score as is the tradition i.e. instrumentation, notation, instructions etc. I decided to make it as a separate file (book)  since the score itself is already quite many pages and therefore big and heavy. It also helps to have the instructions separate since, in my opinion, it adds to the clarity of the work. To keep the music and technical/technological information separated has an aesthetic value.

Sonic analyzes cont. from a blog with the same title.

Measure 37 – 49

After the fermata in measure 36, the Score and the DAW get synchronized before the flute comes in. Here we can easily hear the electronic effects of delay and reverb as the flute continues to play (and sing). The electronic effects get even more audible as the strings and the clarinet enters with an increasing delay, reverb, and granulation. This part is one of my favorites since it sounds very close to the sonority that I had in mind and ads a very elaborate sonic cloud to the chord progression.

The diagram above shows measures 37 – 49 and the connection of the Score to DAW. As can be seen, the volume of the electronic effects increases greatly when the when the multiphonics begin. This comes true when listening to the video recordings below.

It is particularly interesting that the electronic sounds are very clear even though the volume of the instruments, which is the power source of the electronic sounds, is very weak. The main reason is that the effects are very open or strong f (about 85%). It, therefore, seems necessary to increase their strengths very much if low play. It is also unlikely that both the number of acoustics and a large spectrum in the sound will help create this sound-wall of acoustic and electronic sounds.

Synchronization points

Interview with Halldis Rønning (part 3)

Question 10. The written musical Score

The complexity of the Score and writing of electronic sound information?

The use of colors in the Score?

 

The size of the Score – the reduced Score?

Is there a need for visual aid such as iPad screen etc.?

Interview with conductor Halldis Rønning (part 2)

Question 2.   What does the conductor do differently with the music when able to conduct the electronic sound as well as conducting the traditional way? Is there a feeling of fusion of the roles of the conductor and performer?

Question 3.   How does the conductor feel using the ConDiS, does it feel natural toward her way of conducting?

Not hearing the source of her insecurity at the beginning.

And a bit more on that issue… The importance of being able to have more than one concert to get more acquainted with the music, the conducting glove, and the stage setup.

Did a different location of the loudspeakers make a difference? During the “Nordic Tour” performance took place in different “concert” halls with different placements of the ensemble.

Did a different location of the loudspeakers make a difference?

  1. Did a different resonating hall make a difference in the performance? Did a different location of the ensemble make difference for you?
  2. Ensemble located on stage behind the front speakers. (Harpa)
  3. Ensemble located on stage with front speakers on their sides. (Nordic House and Christians Church, Klaksvik)
  4. Ensemble located on stage with front speakers behind. (København)

Did a different resonating hall make a difference in the performance? Probably the piece and the Glove benefits from some space.

 

How can I improve the Conducting Glove “ConGlove”? Page turning!

What musical parameters should the conductor be conducting? Volume, tempo, spatial sound location etc.…

Interview with conductor Halldis Rønning

The following blog is based on an interview I did with conductor Halldis Rønning soon after the Trondheim Sinfonietta, Nordic Tour. Halldis conducted my work, Kuuki no Sukima at concerts in Harpa in Reykjavik, Iceland, Nordic House in Torshavn and Christians church in Klaksvik, Faroe Islands and Schæffergården in Copenhagen, Denmark.

Since the conductor and the art of conducting is one of my focus points in my ConDiS project interviewing Halldis is one of the key points of my artistic research and reflections.

The Interview is based on the following questions:

  • What is your opinion about the idea of expanding the conductor role by conducting the electronic sounds rather than a sound engineer? Is there an Artistic need?
  • What does the conductor do differently with the music when able to conduct the electronic sound as well as conducting the traditional way? Is there a feeling of fusion of the roles of the conductor and performer?
  • How does the conductor feel using the ConDiS, does it feel natural toward his way of conducting?
  • What musical parameters should the conductor be conducting? Volume, tempo, spatial sound location etc.…
  • Is there too much information in the written score (Partitur), too many parameters to control, is it complex or simple enough.
  • During the “Nordic Tour” performance took place in different “concert” halls with different placements of the ensemble.
  1. Did a different resonating hall make a difference in the performance?
  2. Did a different location of the ensemble make difference for you? Can you describe the difference?
  3. Ensemble located on stage behind the front speakers. (Harpa)
  4. Ensemble located on stage with front speakers on their sides. (Nordic House and Christians Church, Klaksvik)
  5. Ensemble located on stage with front speakers behind. (København)
  • Has your opinion of using the ConDiS system changed from the day you were introduced to the technology and its possibilities to now when you have had five performances
  • With more experience and practice using the ConDiS system, has there been any changes in your use of its possibilities?
  • Is there anything that should be added to the ConDiS system or score writing?

Question 1.   The idea of ConDiS? What is your opinion about the idea of expanding the conductor role by conducting the electronic sounds rather than having a sound engineer doing it?

In the above video conductor, Halldis Rønning says “it is a good contribution…to the music field or conductors job” These positive reactions are very important for my research because they mean that Halldis has experienced positive experiences with using ConDiS. Therefore, it can be concluded that my attempt to create a simple device for controlling electronics while managing acoustic sounds have been successful. She did not have much time to take on this instrumental tool but seemed to be quick to catch up with it and perceive his necessity.

Is there an Artistic need?

As stated in the above video interview, Halldis believes that “all integration of electronics and live controlling or having an impact on  the actual electronic sound combined with the acoustic sound is very nice, very fruitful I would say.”  I think this is an excellent proof that ConDiS is a fairly simple and manageable “natural” system, though it may undoubtedly improve it in the future.

The positive respond of Halldis where she agrees with my opinion that there is a need for a new conducting tool and there is an artistic need to expand the role of the conductor led to a question to even expand the role to conducting other media such as video or film.

Controlling Video?

Kuuki no Sukima – sonic analyses

In my blog Closer look at the Score dated 1. March 2018, I did analyze conducting instructions in the opening measures of Kuuki no Sukima. How the conductor had to press the start button (3rd. button) and then the tempo button (4th. button) four times to set the tempo and finally close her fist to activate the volume control.

Sonic analyzes

In this blog, I will focus on the sonic part of the concert, ie. how the sound of the instruments interacts with the electronic sound effects of the computer. I will, as far as it goes, try to showcase the relationship between written notation and a sonic experience. How I use traditional notation in conjunction with “traditional” graphical interface of music software applications. Commonly called “Automation” a control whose value changes in the course of a timeline that is automated.

So, once again let’s take a look at the opening measures of Kuuki no Sukima.

In the opening measures there is no use of electronics and therefore there is a clear dry sound of the instruments until the conductor activates the volume control by closing her fist and raise her arm.  At that point, approximately at the beginning of measure three, the conductor increases the volume of the electronic sounds from cero or no electronics to the level she wants the electronics to be. This is a slight change from the original version where the electronics were supposed to be from the beginning. Why these changes? As mentioned in an earlier blog, this was done to simplify the actions the conductor had to perform at the beginning of the work. Aesthetically it worked out to be a stronger beginning and gave the composition more expressive opening with the electronics fading in with a visual realization or gesture of the conductor raising her arm.

Notated Score and the DAW Interface connection

As can be heard the sound of the Violin start to change in 3rd. measure as soon as the conductor raises her hand. The sounds of the other instruments, the pizzicato with a delay in Cello and Viola. The fast airy note pattern (arco battuto) in the Double Bass, where delay and granulation are increasingly affecting the sound is not as clear. Why?

Lets first take a closer look at the score and the electronic score to figure out how things are connected or related.

The above illustration shows the connection between the notated score and the graphical interface of the computer application or DAW in short for Digital Audio Workstation. Focusing on the Violin part, the computer electronics are added to the high opening note (E).

  1. Delay with diminuendo (decreasing volume) from approximately mf to silence.
  2. Reverb fades in with increasing volume to approximately mp (relatively little reverb)
  3. Granulation starts with ff or very strong and stays unchanged.
  4. Feedback starts with ff (very strong) and stays unchanged.

The above figure shows how the other instruments, Viola, Cello and Double Bass add effects that increase or decrease same way as shown the Violin part. But how come they are not as easily audible as the high Violin pitch?

Fist the loud pizzicato in Cello in measure 3 comes right at the beginning of the conductor’s increase of the electronic volume. Therefore the expected delay effect that is written in the score is not audible. The following Viola pizzicato in measure 4 has an increasing delay and decreasing feedback. It can hardly be heard most likely because the pizzicato is soft (p) and the conductor still hasn’t raised the volume to its maximum value. For the same reason, Double Bass fast airy note pattern starting in measure 4 is not very clear. It should also be mentioned that the volume of the Double Bass is too soft and will be adjusted in next revised version.

The use of electronic effects in Bass Drum and Clarinet entrance in measure 5 should be more audible and closer to the sonic spectrum that is expected. Especially the Clarinet, since although written ppp (very soft) is played louder than for instance the Double Bass. Keep in mind that the Clarinet can´t play the multiphonics very loud and therefore the ppp indicates as soft as possible (should be written in the score). Similarly, the Bass Drum can´t play the finger strike very soft although softer than the Clarinet. The Clarinet multiphonics gives a very rich sound that is high in frequency that should be ideal for picking up electronics, unlike the Bass Drum that has a low-frequency and therefore less audible affectation.

For some reason, that might be related to the choice of effects, the Clarinet multiphonics, and the Bass Drum finger strike seems to pick up very little electronics, much less than expected. The intensity, for instance, Delay and Granular in Clarinet are written ff or very strong that should give maximum effect. It could be my mistake to write a Delay for a sustained note since it can only be delayed at the beginning and I might have to take a closer look at the granulation. For the final version, I will change the Delay effect with a Feedback effect which should give richer sonority as well as adjust the granulation.

Instrument – electronic sound relation.

Here we come to very interesting complications. First of all, the harsh reality of working in the media of mixed music, where there is very often a very little time to work with the performers. It means that tests have to be done by computer simulation. Although I did meet with most of the players during the preparation and composing period I did not have time to adjust the electronic sound effects. Most of the time with the performers was spent working on the extended instrument technique, both the physical aspect and the notation as well as the sonic, how this and that did sound in practice. These are all very time-consuming factors and although the performers were all very positive and helpful there was hardly any extra time for adding the electronics.

Working with Violinist Ina

Same happened during the rehearsal period before the premiere performance in November, most of the time went getting the right acoustic sound without the electronics. It was not until the last rehearsal that the electronics were added with a sigh of release and surprise from the performers. I am not sure why this happens but it seems to happen very frequently in the world of mixed media. One obvious factor is that usually the electronic equipment, loudspeakers, microphones, mixer etc. are not in place until the last minute. But that was not in this case since all equipment was in place right from the beginning. The fact that there were three other works on the program that took significant time to rehearse left too little time. The fact that the conductor is classically trained might have something to do with it. She did spend a lot of time getting the right sound without the electronics, the right balance and other expressions that are important in classical performance. Perhaps the fact that she had to wear a glove to conduct the electronics, the fact that she had to press buttons to move or jump to right markers in the score. The fact that the conducting glove could be more user-friendly. For instance, you could not jump to whatever measure you need to practice or in other words the flexibility of the technology was not good enough. Maybe it was the conductor’s theory, that if the instrumentation was correct then the electronics would be correct a theory that I agreed upon at the beginning. Looking back it might have been a failure since it turned out that the performers did complain that they did not have enough time to learn how the electronics would react towards their performance. The ideal situation would have been a week-long workshop focusing no Kuuki no Sukima only. But that situation is rear and we have to keep in mind that the aim of the research was to create a musical tool for conductors that would be easy enough to use under “normal” situation. Therefore, one could say that the ConDiS prooved to be a successful tool since the conductor did manage to conduct and control the electronics during the performance.

Since all the electronic sounds are totally related to their instrument, there to say the electronic sound of the Violin is based on the sound the Violin is playing at that time. In other words, it is a live real-time sound processing of the Violin. Therefore, if the Violin plays a soft note the electronics are going to be soft and vice versa.  Low frequency its going to sound different than a hight one.

This is problematic since although there is written an ff, meaning a very loud use of the effect, it does not mean that the outcoming sound is going to be loud. It means that the electronic sound is having a large effect on the played tone whether that tone is loud or soft.

In measure 5 the conductor has increased the electronic sound level to the desired point. Keep in mind it is totally up to the conductor to adjust the volume like she does when conducting the “other” instruments. Soon thereafter the Bass Drum and Clarinet start to play. It is possible that the conductor has not adjusted the volume level loud enough but by comparing the three different performances is seems not to be the fact. Let’s look and listen to an example.

Example measures 5-11 from Harpa Concert

Example measures 5-11 from Torshavn Concert

Example measures 5-11 from Copenhagen Concert

As said before here I was expecting more sound processing, especially in the Clarinet.  Why? The Clarinet is playing multiphonics that has relatively rich and high pitch. Therefore the electronic sounds should be clear. It is a bit different with the Bass Drum that is playing a deep drone. Although the multiphonics for the Clarinet is written as very soft (ppp) it is not possible to play them much softer than we hear on the recording. This is also a very good example of the inaccuracy of classical note writing since the ppp means as soft as possible but the relative loudness is more like and mf (mezzoforte) or medium loud.

Other facts that might count is that the conductor’s adjustment of the volume level is not right. The Violin that comes in measure 9 supports the theory because there the conductor has increased even more the electronic sound, and therefore it is much more audible.

Let us now see and hear the opening measures from the Harpa performance with score and illustrations.

Measure 11 – 14.

In measure 11 there is a general pause where the conductor stops the play button of the electronics by pressing the 3rd button. After the pause, the conductor continues by pressing the 4th button which jumps the playback-head of the DAW to exactly the beginning of measure 12.  That synchronizes the DAW and the notated Score so that the Bass Drum and Piano duo should be exactly in sync with the effects of the electronics.

As shown in the figure above Bass Drum comes in with increased reverb and strong feedback effect while the piano has increased feedback, as well as increased —> decreased delay.

Bass Drum – Piano

  1. Reverb crescendo to mf (mezzoforte) medium loud. Feedback with unchanged ff (fortissimo) strong effect.
  2. Delay crescendo to f (forte) —> diminuendo to cero. Feedback from cero to FF (fortissimo/very strong).

Now, look at the automation for the same measures as written in the automation of the DAW.