Appendix

Appendix A

Technical Development

 

Making Technological Decisions

Since the Bridging the Gap – ConDiS project is aimed towards artistic research more than theoretical or technical this paper will briefly mention few important fundamental technical decisions.

 

Hardware

Having decided to make ConDiS as a conducting glove “The ConGlove” was a fundamental decision, to some extent disappointing, since there were high hopes to create something totally “new.” There are dozens of digital gloves out there, many used in virtual reality environments and video but also to directly control audio like Imogen Heap’s “Mi.Mu” Glove.(mi.mu, n.d.)The main difference is, as mentioned earlier that the Bridging the Gap – ConDiS project aims to create conducting tool for music conductor that is based on and adds on to the tradition of classical conducting. Therefore, after much deliberation and experimentation, it was concluded that the most “natural” addition is, after all, to bring the digital control right to the conductor’s fingertips.

 

Sensor

More various types of sensors are now available than ever before, and the prize is getting more affordable. Therefore, one must keep in mind that in the world of this fast-evolving technology what is news today may be yesterday’s news tomorrow.

To select which type of sensor would suit best the idea of the Conducting Digital System; the following criteria was proposed: Wireless technology, simple to use but sophisticated enough to fulfill the needs of being. Reliable, meaning it would not be affected by external interference such as infrared lights or human sweat, comfortable and as natural for the conductor to wear and finally low-cost technology. The following sensors were tested: The Leap Motion Controller, The Myo Gesture Control Armband, The Hot Hand, The Qualisys Motion Capture Systems, Xbox One Kinect 2.0 Sensor and the x-io Technologies, x-OSC sensor.

Decided to use the X- OSC for the Conducting Digital System cause of the following: (see table)

 

 

Brand Tech. type Simple/Complex Reliability Comfort Prize
Leap Motion USB _ _ + +
Myo Armband Infrared + _ + +
Hot Hand Bluetooth _ + + +
Qualisys Infrared _ + _ _
Kinect 2.0 Infrared Software released just recently (Feb. 2017) for Mac.

Not enough time to test

+ +
X – OSC Wi-Fi + + + +

Table 1. Evaluation of sensors. + means positive result, – means a negative result.

Software

ConDiS is intended to be accessible to others without the presence of the developer. It is intended to be flexible and open for personal adjustments and or individual experimental developing, an essential fact in the artistic philosophy of the Bridging the Gap – ConDiS project. Therefore, the decision was made to use Max/MSP, Max for Live and Ableton Live, a commercially available software that is user-friendly and yet flexible enough to fulfill the artistic needs of the project.  This software is all commonly used by composers, performers, software designers, researches, and artists to create various forms of artistic performances and installations. All an important factor in choosing the right software.

Basic motion test of the x OSC Wi-Fi Board.

The following is a brief description various test made to understand and get the “feel” for the usability of the x-OSC Wi-Fi board. It is by no mean a thorough technological research, rather a practical research aimed to understand the possible artistic use of the x-OSC Wi-Fi board.

Traditional Conducting gestures

The basic traditional conducting gestures are gestures indicating the desired volume, meter and tempo. With hand gestures, the conductor traditionally indicates an increase or decrease in volume by lifting up his arm(s) or making larger (for louder) and smaller (for softer) gestures. With different arm gestures, the conductor indicates the written time signature, and with a variable speed of the arm gestures, the conductor indicates the written tempo or metronome. To find out if there was a possibility to use the x-OSC sensor for recognizing and learning various traditional conducting gestures the following study test was made.

 

Up/down motion

Volume

Holding hand open and raising arm up and down (palm facing the ceiling on the way up and floor on the way down) I got the following patterns:

Figure 65.  Arm slowly up /down. Peaks at turning points.

 Raising hand slowly as shown in Fig. 15 the X-axis (red) indicator showed a gradual move in the direction of the arm, moving up and down. Turning points are also traceable in the form of high and low points. The Y axis (blue) showed a small but gradual movement in the direction of the arm but apparent peaks at turning points. The Z axis (green) showed an up/down movement when the arm was raised slowly and down/up motion when lowering the arm. Fast move up or down was shown at turning points.

Figure 66. Arm fast up/down. No peaks at turning points.

When the arm was raised quickly up and down (Fig. 6) a slightly different picture was shown especially at the Y and Z axis, while X-axis was mostly identical. The only notable difference being minor deviation especially at a high point, probably caused by computer latency. The Y-axis showed an up/down motion when the arm was raised and down/up motion when lowering the arm with no peaks at turning points. The Z-axis showed up/down motion on the way up and down/up motion on the way down with no peaks or change of direction at turning points.

4/4 beat conducting gesture – Beat/Tempo

 

Conducting standard 4/4 beat gestures in two different tempos fast and slow (metronome 60 for slow and 120 for fast), I got the following patterns:

 

Figure 67.  Metronome 60. Counting clear 4/4 pattern.

As shown in Figure 17 all the axes X, Y, Z showed traceable patterns when conducting precise 4/4 beat pattern at a relatively slow tempo, 60 bpm. The X-axis was peaking at every beat (1, 2, 3, 4) while the Y-axis showed reverse motion. The X-axis shows each beat as peak points that are quite clear. Y-axis shows reverse motion with peak points at upbeat to first, second, third and fourth beat and a definite low point on the first beat. The Z axis is somewhat clear showing peak points at every beat although first and second beat is wholly unclear.

Figure 68. Metronome 60 counting less clear pattern.

When conducting the same tempo with a bit “natural” style (Figure 18.), i.e. very clear downbeat and the rest 2nd, 3rd, 4th, beat, not as strict or more flowing as in the first example the patterns became less clear especially the 1st beat could easily be confused with the 4th beat.

 

Figure 69.  Metronome 120. Counting clear 4/4 patterns

When conducting clear beat in a relatively fast tempo of 120 bpm, the patterns became much more blurred. As shown in figure 19. The upbeat to first beat was still there, especially on the Y-axis. X-axis showed as before peak points at 1st and 2nd beat though much blurrier than before. Z axis was as previously somewhat obscure and irregular.

Figure 70. Metronome 120. Counting unclear 4/4 patterns

When conducting in more fluid gestures still with strict upbeat and downbeat to the first beat, the first beat was clearer but the whole pattern became blurrier. This happened since the up/down beat to the 1st beat was accented while the other beats (2nd, 3rd, and 4th) were less or even not accented at all.

 

Conclusion

As expected conducting straight and clear 4/4 pattern gave a clear result as repeated patterns with peaks on each beat. When conducting more freely the patterns started to get blurry. Same happened with increased tempo, faster tempo resulted in less predictable patterns. Having the MuBuForMax – hhmm program to learn the conducting patterns resulted in precision between 50 – 80% a result not satisfying the artistic goal of the ConDiS system.

It is a fact that conductors like to use expressive conducting gestures for conducting tempo, meaning being able to use patterns that go beyond the strict metric gesture. They are musicians and they need to move them self’s freely and as conductor Halldis Rønning clearly states in the enclosed interview.

Results for the conducting 4/4 beat revealed that it was impossible for the above reasons to use the MuBuForMax – hhmm learning feature. For the conductor to be able to conduct tempo as freely and musically other solutions had to be found.

 

Circular motion

Although circular motion is not a part of the conductor’s vocabulary, it was necessary to find out if it would be useful for the original idea of having the conductor controlling the panning of the electronic sound. The idea that if the conductor made a significantly different gesture from the traditional one, there would not be any confusion between the conductor and the performers. Circular motion, therefore, was probably the nearest one, since hardly used as a traditional conducting gesture.

 

Panning

 

The final test was to measure circular motion, full circle, and half a circle all gesture that might be useful for locating and move sound in space (between loudspeakers). The first motion to calculate was circular motion with arm right up above head and moving clockwise around the head.

Figure 71. Left-arm overhead moving 90° pr. sec.

Circular motion left hand over the head, the palm facing right moving one circle 360° pr. 4 sec. or 90° pr. 1 sec. or equivalent to the tempo of 60 bpm. As can be seen in figure 21, the pattern is very clear and linear on all the axes. The x-axis shows the high point at 0°and the low point at 180°. The Y axis has high at 180° and low at 0° or a phase of 180° from X-axis. The Z point is in +- 90° phase from the other axes with high points at 90° and low points at 270°.

 

Controllers

Substantial time was spent getting the software together. Early in the process, I traveled to Ireland to meet with my co-supervisor Mikael Fernstrøm Director of the Interaction Design Centre at the University of Limerick. During the stay, we shared information and knowledge concerning design and technology. The main topic being the importance of designing a convenient and wearable conducting hardware. Mikael named a smartwatch as a potential object to develop further for conducting. Conducting baton, glove and spectacles were also discussed as well as motion tracking devises. Coming back from my stay with Mikael it was clear or at least clearer that the conducting tool (hardware) had to be a single unit designed as a logical extension of the conductor, not unlike the conducting baton.

Mostly programming in the Max/MSP environment and then transferring it to M4L and Ableton Live. It was important to have ConDiS system running correctly, to be able to test the artistic results initially aimed for, i.e., interactive remote control of electronics through conducting gestures.

Appendix B

The ConDiS hardware

Designing the hardware

One of the first ideas of the ConDiS hardware or tool was to extend the use of the conductor’s baton by making a digital pole/baton. A prototype of a somewhat clumsy looking baton was made various gestures recorded to find what kind of patterns it would make.

The ConBaton, (first prototype).

To measure the conducting gestures, the MuBu multi-buffer program from IRCAM running Max/MSP on OSX was used. (Ircam, n.d.)It is a multi-track container for sound description and motion capture data or Probabilistic Models for Designing Motion and Sound Relationships. The type used for the ConDiS motion analysis was; mubu.hhmm (Hierarchical Hidden Markov Models) based on the hhmm_leapmotion_recognition program by Masayuki Akamatsu (Akamatsu, n.d.)but slightly modified for the use of x-OSC Wi-Fi board.

Conducting Glove

Figure 72. The Conducting Digital Glove “ConGlove”.

Conducting Digital System – ConDiS consists of a glove controlled by x-OSC wireless I / O board that communicates over Wi-Fi using open sound control[1](OSC). Software based on Max / MSP sends information from glove through Max4Live to Ableton Live digital audio workstation. The conducting glove is designed to enable a conductor not only to control the overall mix of the performing musicians and electronics but also to synchronize the performers and associated DAW.

In other words, conducting the overall balance/volume, tempo and synchronize written score and DAW at selected cue-points written in the score.  ConDiS is directed toward new possibilities in musical composition and the interaction, the expressions, the musical gestures and movements of the classical conductor.

Functions of the ConGlove

Bending sensors

There are five bending sensors “built in” the conducting glove. One for each finger allowing the use of sign language to send messages to the DAW. For instance, by sending out message holding out the thump the DAW reads the message as value of one (1), holding out index finger as value of two (2) holding out middle finger as value of four (4), ring finger as value of 8 and little finger as value of 16. Any combination of fingers would then be the sum of the fingers, for example, holding out a thumb and index finger would give a value of 1+2 = 3 or combination of index and middle finger (a peace sign) would give a value of 2+4 = 6. The bending sensors are then scaled to give a value from 0 – 127. 0 indicating fully bend finger and 127 for fully stretch out a finger.

 

OSC x-I/O Wireless Sensor

The conducting glove comes with x-OSC wireless I / O board that communicates over Wi-Fi using OSC. It provides up to 32 analog / digital channels and equipment with three onboard sensors; gyroscope, accelerometer, and magnetometer. For the ConDiS system, I decided to use the accelerometer to measure the movement of the conductor’s arm from 0 as the lowest possible position and 127 the max.

 

Knobs

There are four knobs on the conducting glove, one for each finger except the thumb. It being excluded since the thumb needs to trigger the other knobs. Each knob triggers an on/off message with 0 as the “on” value and 1 for “off” value. These knobs are used to synchronize and indicate tempo and cue-points between the DAW and written score

 

Appendix C

Conducting Gestures and Signs

 

Gestural Functions

The original idea was to have the conductor’s role a bit larger by allowing her to control electronic sonority of individual instrumental groups as well as give her control of other features. That including panning of the sound (move the sound in space) and the sound timbre (sound color) by opening and closing spectral filters, changing reverb and delay time, altering envelopes, etc.

All these controls are available; the conductor can control individual instrumental groups by using signs of one, two, three, four and five fingers (see blog Notation and control from March 13, 2017).

The panning or moving sound in space can be done by moving or twisting the arm wearing the conducting glove, ConGlove. The change of sonority can also be made by raising the arm wearing the ConGlove (see blog Controlling using Conducting Gestures from October 3rd, 2017).

 

Gestures

It is important to make the additional conducting gestures for the computer control somewhat “natural” and simple. Therefore, for the control of “levels” such as Volume and Effect the conductor should raise or lower her arm for increased or decreased levels. When it comes to panning or how the conductor should be able to steer the spatial location of the sound, the answer is not as obvious. First, an attempt was made to have the conductor raise her/his arm into the air and then move it into circles. That turned out to be a bit “cowboy” like gesture and therefore disturbing and not fitting my artistic goal of “natural” gesture. Next, the attempt was to have the conductor holding arm straight out and then move hand in half circle (the whole circle not physically possible).

Figure 73. Wearing the “ConGlove.”

It was a bit more natural but still did not feel entirely right. Now the emphasis is on a solution that at the moment seems to be close to what was aimed for and that is to have the conductor moving the elbow in a circular movement. When trying it out, it gives the right natural feeling and also the possibility to move hand in a diagonal direction as well as a circle.

ConDiS allows the conductor to control three primary functions. The level of the outcoming computer-generated sound, the spatial location of the sound (pan) and the amount of effect added to the sound. A decision has been made to trigger these functions by using sign language, an OK sign for activate Volume control, Little-finger up for activating Effect control, a Thumb-up for Pan and closed hand for all control off.

 

Controllers and signs.

Control

Controlling using Conducting Gestures

As part of the Bridging the Gap – ConDiS project, testing various conducting gestures to control electronics and the balance between live electronics and live acoustic sounds have been crucial. Answering the questions on how to add these controls to the professional music conductor’s gestural library? Is it possible to use classical conducting gestures only, or is there a need to invent some new?

To use already well-known gestures is the safest way. The main reason is purely practical. The professional orchestral conductor has trained for years his musical gestures, and it might be complicated to ask her to learn a new one. Why not use the same gestures since the conductor is conducting very similar elements in the electronic as for the acoustic instruments? Therefore, making the use of ConDiS less noticeable or invisible. For that reason, the decision was made to use the following gestures all well known in the conducting repertoire.

The original plan, made at the start of the ConDiS artistic research program, the controllers (parameters) to be controlled by the conductor were grouped into the following categorization:

  1. Overall volume.
  2. Spatial location.
  3. Sonority and spectral timbre.
  4. Conducting tempo (tap tempo)
  5. Synchronization between conducting score and electronic score.

 

All these factors have been realized, and function as shown in the following video examples.

 

The following is based on video recordings made as instructions for the ConDiS controlling usage possibilities. It should be noticed that, as mentioned before, the panning and effect control was later left out from the final version of ConDiS.

 

  1. Volume control (Overall volume).
Figure 74. Video example. Volume control (video demo)

http://caveproduct.com/videos/Vol.-control.mp4

The conductor can raise or lower the overall volume of the electronic sound. With simple finger gesture, an OK sign, the conductor can trigger the volume control feature “on” and then by lifting left arm raise the sound. When the volume level is at “right” level, the volume control is triggered “off” by closing the hand. Same goes for lowering the sound except the arm must be lowered. (See Figure 74).

The use of a traditional up/down arm gesture to indicate an increase or decrease of sound levels gave a positive artistic result. It was well suited to the traditional conducting gestures for

 

  1. Pan control (spatial location).
Figure 75. Video example. Pan Control (video demo)

http://caveproduct.com/videos/Pan_control.mp4

The conductor can move sound in space. He can with finger gesture (thumb up) trigger the pan control feature and then by tilting the hand move the location of the sound. As with volume control, the pan control function is deactivated (turned off) by closing the hand. (See Figure 75).

 

 

  1. Effect control. (sonority and spectral timbre)
Figure 76.Video example. Effect control (video demo)

http://caveproduct.com/videos/Fx_control.mp4

The conductor can raise or lower the overall effect volume. He can with finger gesture (little finger out) trigger the effect control feature and then by lifting or lowering left arm raise or lower the sound. The effect control function is turned off by closing the hand. (See Figure 76).

 

  1. Conducting tempo (tap tempo)
Figure 77. Video example. Tempo control (video demo).

http://caveproduct.com/videos/Tempo_Control.mp4

The conductor changes tempo by clicking a knob on the middle finger accordingly to a written tempo. The tempo controller can be set to calculate the average time between any number of clicks. That means it can calculate the time between two clicks minimum to an infinite number. If the score is in 4/4 a calculation setting of 4 would be the most natural setting although a setting of 2 can efficiently work. For pieces with frequent meter changes a setting of 2 seems to give the best result. (See Figure 77).

 

  1. Cue control. (Synchronization of conducting score and electronic score)

 

The conductor can, by clicking knobs on the ring finger and little finger move the electronic score back and forth to selected numbers (rehearsal numbers) written in the score.

Figure 78. Video example. Cue control (video demo) Video.

http://caveproduct.com/videos/Jump to cue.mp4

 

 

Expression

As can be seen clearly in the above videos, the emphasis was on making use of the conducting glove as normal to the conductor´s gestures as possible. With this, the conductor could focus on his traditional way of conducting with the same natural emotion and expression to which she is accustomed. It can easily be argued that pushing knobs is not a conducting tradition or a very expressive way of conducting. These arguments can surely be accepted, but this was the simplest solution in the situation where gesture recognition technology was not reliable enough. By using knobs in the conducting glove, there is no holding back or at least as little as possible the “natural” movements of the conductor. That way the conductor can continue hers conducting gestures with all the associated musical feelings and expressions.

 

 

Appendix D

Performance preparation

 

Synchronization between Score and DAW

In order to understand better the functionality of the ConDiS system, both for practical and artistic use, a closer look at the Score and DAW synchronization are necessary.

Starting with the performance preparation of the first version of Kuuki no Sukimaused at the trial performance at Dokkhuset and later compare it with the last version of the score gives an excellent insight to the process that took place in the further development of the project.

A sequence of short videos is used to make it easier to follow and understand the subject.

Kuki no Sukima – Pre-concert preparation

Before entering the stage, the conductor needed to prepare the performance by performing the following actions:

 

  1. Activate the x-OSC sensor by turning on the on/off switch located on the wrist of the conducting glove.
  2. Press the 3rd finger knob once to stop the playback function that happens when the sensor is turned on.
  3. Press the 5th finger knob twice or more to set the DAW play-head to the beginning.
  4. Select the OSC network (needs to be done on the laptop).
  5. Activate the calibration toggle on the ConDiS interface. Open and close hand three times for finger recognition. Deactivate calibration toggle.

Now all is prepared for the conductor to walk on the stage.

 

Figure 28. shows the opening page of the first version of Kuki no Sukima – Between the air. Instructions are written and indicate what the conductor needs to do in the opening measures of the piece.

 

Figure 79. The opening page of Kuuki no Sukima 1st. movement. The first version of the composition as performed at Dokkhuset.

When on stage the conductor needs to do the following actions:

  1. OK sign to activate the Volume interface of DAW
  2. Click the 2nd finger knob three times to set the tempo
  3. Click the 3rd. finger knob to activate play-head knob of the DAW
  4. Close her hand to activate the Volume Value Control
  5. Move arm up or down to set the Volume Value
Figure 80. Ok, sign.

First, the conductor needs to give an OK sign with left hand wearing the Conducting Glove (ConGlove). It tells the DAW to activate the Volume control unit of the ConDiS interface.

After activating the volume (OK sign), the conductor has to give the written tempo by clicking her index finger (finger no.2) three times. As indicated with;

 

Figure 81. knob 2 = set metronome

That sets the metronome of the DAW somewhere close to the given tempo (metronome 48), depending on the conductor’s accuracy.

Now, the conductor is ready to start the performance by pressing knob three.

Figure 82. Start performance

Figure 83. Closed hand

The conductor needs to close her hand to activate the volume control levels and raise her arm a bit. That will set the volume level of the electronics.

 

ConDiS Graphical Interface

Figure 33 and 34 illustrate the action of the ConDiS digital interface when activated.

 

Figure 84.  The ConDiS system de-activated. Red frame showing the volume-selector with sliders at volume 0.

 

Figure 85.The ConDiS system activated. The Acceleration meter activated, finger calibration on, tap tempo and metronome set and volume levels up (turned red).

 

From left to right –

  1. Accelerometer, to calculate arm position.
  2. Vol_Pan_Fx_selector, that indicates finger positions from little finger slider-values to the left to thumb too far right. When given an OK sign the index finger, and thumb finger values go down.
  3. Tap Tempo – Calculates the average time between 2ndfinger knob clicks. In this example, number 3 is selected for average metronome of four clicks (0-3). Any number can be set for tempo calculation.

 

This preparation turned out to be complex and confusing for the conductor. She felt there were too many actions in preparation for start conducting. Therefore, before the first performance at Dokkhuset, the volume control and metronome were manually activated, eliminating the OK sign and pressing metronome knob. Therefore, she only needed to press the third knob once to start the piece.

That was a last-minute compromise and needed correction before the next performance. It suffered the tight synchronization that was supposed to be between the played instrumental sound and the live electronic sound since her conducting beat was significantly faster than manually set for the DAW. Since there are spots in the written score (marked with knob 4), where the conductor synchronizes the score and electronics it never went far off. Still, it hurt the performance.

How to start performing – instruction videos

The following are seven short instruction videos made as follow up to the previously written text. They are based upon an earlier version of the musical score and should give a good insight into the complication of starting the performance. With a focus on the opening measures, the video clips give a good overview of the conductor’s job and facilitate understanding of the functionality of the ConDiS system.

 

N.B. The musical score used in these videos is from an older version of Kuuki no Sukimaand, therefore, looks slightly different than the final version. The main difference being less information to the conductor.

 

Instructions for the Conductor:

Video example #68: http://caveproduct.com/videos/KnS_Cond_ins_part….mp4

A bit closer look at the instructions:

Video example #69: http://caveproduct.com/videos/KnS_Cond_ins_part….mp4

After the conductor has activated the volume control, she can adjust the volume value of the electronic sound by moving her arm up and/or down.

Moving arm up and down:

Video example #70: http://caveproduct.com/videos/KnS_Cond_ins_part….mp4

Few other extra things the conductor needs to do before starting the piece.

The conductor has to set the metronome for the piece which in this case is metronome 48. To do so, she needs to press the index finger knob four times in the tempo of the written metronome. NB. The instructional video uses an old version of the musical score.  Hence, the narrator and the musical score does not always coincide.

Setting the metronome:

Video example #71: http://caveproduct.com/videos/KnS_Cond_ins_part….mp4

After setting the metronome, the conductor has to press the 3rd. knob to tell the computer that she has set the metronome.

The following video explains in more details what the conductor has to do in the opening measures of the Kuki no Sukima.

Opening measures of Kuuki no Sukimaat Dokkhuset concert.

Video example #72: http://caveproduct.com/videos/KnS_ConBeg.mp4

However, the aim of the project is to create a system simple enough for conductors to use as a “natural” extension of their conducting tradition. Therefore, simplification was needed before the next scheduled concert, The Nordic Tour.

 

Kuuki no Sukimathe Nordic Tour version

 

Pre-concert preparation

The following pre-concert preparation needs to be done before the conductor enters the stage (unchanged from Dokkhuset performance):

  1. Turn the sensor on by clicking the on/off knob attached to the sensor.
  2. When the network is turned on for some reason, the DAW starts playing. Therefore, the conductor has to press the 5th knob four times to move the play head back to the beginning and stop playing.
  3. The conductor has to press the 3rdknob once to stop the rewind function of knob five.
  4. Select the OSC network (needs to be done on the laptop).
  5. Activate the calibration toggle on the ConDiS interface. Open and close hand three times for finger recognition. Deactivate the calibration toggle.

These preparations are unnoticeable for the performers and the audience since they are all done before the conductor enters the stage. Still, they are hassling and stressful.

Changes made from the first performance:

Activating the volume control unit is no longer necessary since there are no other units to activate after leaving out pan and effect control.

 

Figure 35. shows the opening page of the first version of Kuki no Sukima – Between the air. Instructions are written and indicate what the conductor needs to do in the opening measures of the piece.

Figure 86. The opening page of Kuuki no Sukima 1st. movement. The second version of the composition as performed during the Nordic Tour.

 

New order to start the performance:

  1. The conductor gives a start playing sign by clicking the 3rdknob on the first downbeat, not the upbeat as in earlier performance.

This solution worked very well causing no problem starting the performance synchronized right at the beginning.

  1. The conductor gives the metronome speed after starting the performance instead of before as in earlier performance.

This change worked very well being a natural extension to her conducting technique.

Figure 87. Press 3rd. knob to start performance. Press 2nd. button to set metronome (tempo 48). Illustrating showing relationship between the musical score and DAW.

 

  1. The conductor closes her fist and raises the electronic volume level in measure three instead of doing it at the very beginning.

Figure 88. Start performance, set metronome, adjust volume.

 

  1. When the electronic volume value has been set the conductor deactivates her control by open up her fist as shown in the next example:

Figure 89. Control of Volume value deactivated

This solution worked very well the computer recognized the close/open fist gesture faultlessly, and there was an audible increase in volume when the conductor raised her arm.

This solution seemed to work very well since it:

  1. gave the conductor more confidence in adjusting the electronic volume
  2. gave the opening a bit more breath or space to open up
  3. showed clearly the use of ConDiS
  4. aesthetically more graceful opening

Confusing not to see the volume value.

Conversations with conductor Halldis after the concert in Harpa, however, revealed that she was not entirely confident not knowing how much she had increased the volume. Especially uncomfortable was not to know how much headroom was left, i.e., whether she could increase the volume even further. Let’s take a look at the interface:

Figure 90. Volume selecting interface. 0= no volume 0.5= half the min/max volume value, 1. = max volume value

The first Volume selector shows the volume value at zero points; the next one shows volume value at half point, 0.5 and the last one shows the volume value at its maximum point of 1.0.

Suggested placing this visual information on the stand next to the score in the form of iPad or iPhone. The conductor replied after some thoughts that although sounding like a good idea it might be too distracting for her concentration conducting the score.

Later during the Nordic Tour, we found out that it was essential to locate the conductor at the exact right spot on stage, where she could hear the electronics and the ensemble equally clear. That way she had no need to see the Volume value interface.

 

Appendix E

List of Performances

The following is a list of performances and lectures given during the research period.

 

Research Concert at Dokkhuset, Trondheim.

This hour and a half long video almost say´s all needed about my Conducting Digital System ConDiS.

 

Video example #73: https://vimeo.com/266997352/6cc5700384

 

Research Concert at the Organ room Olavshallen, Trondheim.

An eight-voice version of the “Hljóðs bið ek” for 14 voices was performed at the Olavshallen Organ room in in Olavshallen June 24th, 2015. Result started an early thinking about the importance of a simple graphical interface. Forgetting to press an on/off button had me conducting without any response from the system. Found out very late in the performance, too late, to make it a successful demonstration. Still for me a valuable lesson for further process.

 

Video example #74: https://www.youtube.com/watch?v=SNPb1lcddnE

 

 

Test Concert at Dokkhuset, Trondheim, Norway.

Performed at the Virtuoso Listener festival in Trondheim November 24th. 2017

The Dokkhuset concert was a typical trial concert testing out various technical and practical elements. Partly since I was still looking for the right sonority based on my extended notation experimentation. During the rehearsal period, substantial time was, therefore, spent rehearsing the ensemble without the electronics. For that reason, the electronics did not come into the rehearsal process until the last day of rehearsal. Therefore, not leaving enough time for the performers to digest the sonic extension to their instruments as well as the overall sound of the ensemble. Neither did it give conductor Halldis enough time to study the possibilities and musical advantages of the conducting system. The rehearsal time that I thought would be plenty happened to be not enough.

 

The concert at Dokkhuset was unsatisfactory though not in a performance sense but rather concerning overall sound quality.

 

Kuuki no Sukima1stmovement.

Video example #75: https://www.youtube.com/watch?v=V_03oqu0kpU

 

Kuuki no Sukima2ndmovement

Video example #76: https://www.youtube.com/watch?v=UM2ULIQixNw

 

Kuuki no Sukima3rdmovement

Video example #77: https://www.youtube.com/watch?v=dvqH_ATZkaQ

 

Trondheim Sinfonietta – Nordic Tour

Video Recordings

The following websites include video recordings during the Trondheim Sinfonietta – Nordic Tour. Four concerts in six days, in Harpa Music Hall, Reykjavik, Iceland. Nordic House, Torshavn, and Christians Church, Klaksvik, in the Faeroe Islands and final concert at Schæffergården, Copenhagen.

 

Concert in Harpa, Reykjavik, Iceland

Trondheim Sinfonietta first concert during the Nordic Tour. Recorded at the Harpa Music Hall in Reykjavik.

January 27. Myrkir Musikdagar festival (Dark Music Days), Reykjavik, Island

 

Kuuki no Sukima1stmovement.

Video example #78: https://youtu.be/xlU_911ILKo

 

Kuuki no Sukima2ndmovement

Video example #79: https://youtu.be/QbD7eqD4MiQ

 

Kuuki no Sukima3rdmovementVideo example #80: https://www.youtube.com/watch?v=JWomugjE74c

 

Concert at the Nordic House, Torshavn, Faroe Islands

January 30th. Nordic House, Torshavn, Faroe Islands

 

Kuuki no Sukima1stmovement.

Video example #81: https://www.youtube.com/edit?ar=1&o=U&video_id=8MS6FtlDxys

Kuuki no Sukima2ndmovement

Video example #82: https://www.youtube.com/edit?ar=1&o=U&video_id=kyWi0hQANnk

Kuuki no Sukima3rdmovement

Video example #83: https://www.youtube.com/edit?o=U&ar=1&video_id=_waZflAFWb0

 

Concert at the Christians church, Klaksvik, Faroe Islands

January 31st: Christians church, Klaksvik, Faroe Islands

Kuuki no Sukima 3rdmovement

Video example #84: https://www.youtube.com/watch?v=KgA8gGaMzzg

 

Concert at Schæffergården, Copenhagen, Denmark

February 1st: Schæffergården, Copenhagen

Kuuki no Sukima1stmovement.

Video example #85: https://www.youtube.com/edit?o=U&ar=1&video_id=UTjQRJqEqzk

Kuuki no Sukima 2ndmovement

Video example #86: https://www.youtube.com/edit?ar=1&o=U&video_id=RA9aagwcUB4

Kuuki no Sukima3rdmovement

Video example #87: https://www.youtube.com/edit?o=U&ar=1&video_id=2RD0mvb6o7E

 

 

Appendix F

Questions and answers – performers.

 

Marianne Baudouin Lie, cellist:

  • What is your opinion about the idea of expanding the conductor role by conducting the electronic sounds rather than a sound engineer? Is there an Artistic gain?

 

I think that it feels more like the electronics are also a part of the performance within the ensemble, maybe more than if it was steered from outside. It also gives extra tension as to whether or not everything works, and I find myself listening to hear if the electronics are there. I do get a bit distracted by the glove and worrying about it working, but that might be because this is a prototype and things have in some instances not been working, or the levels seem uncontrollable. It is almost so I would have wished for the composer to have a small console to see how the levels were behaving.  But at the same time, I didn’t think the distraction made the performance less concentrated, it was more like an observation.

 

  • With the conductor being able to control the electronic sound as well as the acoustic. Does that change your perception of the conductor’s role?

 

Well, it makes me pay attention to the things she does with the glove and I’m impressed that she has learned to use it as well as concentrating on everything else in the score.

 

  • How does it feel watching the conductor using the ConDiS? Is it “natural”, confusing or distracting when she moves the conducting glove.

 

It is somehow a bit distracting with the glove because I keep wondering if anything is going to go wrong, or if she will push the right knob. Or what happens if the battery actually falls off etc. I’m also very aware that she has to change her main conducting hand, but this does not bother me.

 

 

  • Should additional information about the electronics be written in your musical parts such as reverb and delay? Is less more? Anything else?

 

I do not feel that I need more information about the electronics as long as I’m not in charge of steering anything.

 

 

  • During the “Nordic Tour” performance took place in different “concert” halls with different placements of the ensemble.

 

  1. Did a different resonating hall make a difference of the performance?

Yes, I think so, or maybe it was that the electronics were given more reverb. I felt like a room with more reverb created a better unity of the acoustic and the electric sounds.

 

  1. Did a different location of the ensemble make difference for you? Can you describe the difference?
  2. Ensemble located on stage behind the front speakers. (Harpa)

Here I could hardly hear the electronics and it felt a bit like not being part of the work, like just being “the ensemble” and not inside the music. I was also uncertain as to whether the audience were hearing enough electronics.

 

 

  1. Ensemble located on stage with front speakers on their sides. (Nordic House and Christians Church, Klaksvik)

 

This worked better, but I still felt that either the levels were a bit low or that I just couldn’t hear the electronic part. But I know Rasmussen was very happy with the work in Klaksvik and thought the acoustic in the hall and the electronics created a good unity.

 

  1. Ensemble located on stage with front speakers behind. (Copenhagen)

 

Here we felt like a part of the entire work, and I think this affects the way we play and the way we imagine the music. So, I think this is the best location of the speakers – keeping both the ensemble and the audience inside the circle of sound. I think it is then also easier to give more as a performer.

 

  • Can you describe your personal experience during the concert performance?

 

I had difficulty trusting the sound I was hearing, which is difficult being an acoustic musician used to orientate everything from what I hear around me. This felt better when the ensemble was within the circle of sound. I could also not let go of worrying about something not working, but maybe it is always like that with acousmatic/ mixed music? I enjoyed playing when I could hear the effect the music was creating, feeling like a small part in a big mix.

 

 

Peter Hatfield, French Horn player:

 

  • What is your opinion about the idea of expanding the conductor role by conducting the electronic sounds rather than a sound engineer? Is there an Artistic gain?

 

I think it has possibilities. We talked about Boulez “Répons”, but I´ve only heard that on TV or on CD, never live, so I´m not sure how reliant Boulez was on his sound man. As Halldis, or any conductor did this or any other piece involving the glove, I guess they would become more creative with it.

 

 

  • With the conductor being able to control the electronic sound as well as the acoustic. Does that change your perception of the conductor’s role?

 

Not really. The actual “Conducting” part of the job was relatively simple: If one found a way to use the glove in Répons, which I far more complicated, with many fast time changes (5/16 – 7/16 – 3/8 etc) it might change rather more.

 

  • How does it feel watching the conductor using the ConDiS? Is it “natural”, confusing or distracting when she moves the conducting glove.

 

This is a continuation of the previous question. There is a danger of mistaking an expressive gesture in the left hand for something to do with the electronics and vice versa. In “Kuuki” it wasn´t really a problem, but I can imagine it might be in a different piece.

 

  • Should additional information about the electronics be written in your musical part such as reverb and delay? Is less more? Anything else?

 

On stage, I didn´t hear the electronics as much as I expected to (at least in comparison to the Harpa performance on You Tube. I was pleasantly surprised when I heard anything, so I don´t think for myself I need more information than I had. There is a danger of “Too Much Information”.

 

  • During the “Nordic Tour” performance took place in different “concert” halls with different placements of the ensemble.

 

  1. Did a different resonating hall make a difference of the performance?

 

I became less and less aware of the electronics as the tour went on. Even more than most performances, the stage is not always the best place to judge the overall effect of a performance. I got the impression that you and Haldis were more careful with the volume levels in the smaller halls due to the danger of feedback, so the overall effect seemed to be less integrated.

 

  1. Did a different location of the ensemble make difference for you? Can you describe the difference?
  2. Ensemble located on stage behind the front speakers. (Harpa)

 

 

  1. Ensemble located on stage with front speakers on their sides. (Nordic House and Christians Church, Klaksvik)

 

 

  1. Ensemble located on stage with front speakers behind. (Copenhagen)

 

One would have expected to hear more of the electronics with the speakers behind the ensemble, but I hardly noticed it at all there (volume levels too low?)

 

  • Can you describe your personal experience during the concert performance?

 

As I mentioned it was difficult to get the whole picture from where I was sat. For myself I was pre-occupied with what I had to do personally and was pleasantly surprised (but I hope not distracted…) when I became aware for the electronics, especially when they were generated by something I had played.

 

 

Trine Knutsen, Flute player

 

  • What is your opinion about the idea of expanding the conductor role by conducting the electronic sounds rather than a sound engineer? Is there an Artistic gain?

 

Spennende tilleggsfunksjon for en dirigent. Bidrar til at framførelsen blir unik fra gang til gang.

Translation.

Exciting additionality for a conductor. Continues to make the performance unique from time to time.

 

  • With the conductor being able to control the electronic sound as well as the acoustic. Does that change your perception of the conductor’s role?

 

Hun får mer å holde styr på, flere muligheter til å gjøre feil osv. Dvs ikke nødvendigvis en positiv tilleggsrolle.

 

Translation.

She gets more to keep track of, more opportunities to make mistakes, etc. That is not necessarily a positive additional role.

 

  • How does it feel watching the conductor using the ConDiS? Is it “natural”, confusing or distracting when she moves the conducting glove.

Mest forstyrrende egentlig. Særlig fordi hun stadig kommenterte at hun egentlig ikke hadde kontroll på volumet, dvs ikke følte at hun faktisk hadde kontroll på effektene når hun brukte hansken.

 

Translation.

Most disturbing really. Especially because she constantly commented that she did not really control the volume, meaning she did not actually control the effects when she used her glove.

 

  • Should additional information about the electronics be written in your musical part such as reverb and delay? Is less more? Anything else?

Ja gjerne, Fint å vite hvilken effekt som er tilsiktet. Slik kan musikeren også bidra til å optimalisere effektene.

 

Translation.

Oh, yes, Nice to know what effect is intended. This way the musician can also help optimize the effects.

 

  • During the “Nordic Tour” performance took place in different “concert” halls with different placements of the ensemble.

 

  1. Did a different resonating hall make a difference of the performance?
    • Først og fremst fordi man tilpasser seg akustikken ift egen dynamikk, artikulasjon osv.
    • First and foremost, because you adapt to acoustics in your own dynamics, articulation, etc.

 

  1. Did a different location of the ensemble make difference for you? Can you describe the difference?
  2. Ensemble located on stage behind the front speakers. (Harpa)

Opplevdes som at man spilte mer separert, alene. Akustikken i denne salen var “sterk”, dvs at man kunne spille veldig svak dynamikk og effekter og at det fortsatt bar godt til publikum i salen.

 

Translation.

Experienced as playing more separated, alone. The acoustics in this hall were “strong”, meaning that you could play very weak dynamics and effects and that it still benefited the audience in the hall.

  1. Ensemble located on stage with front speakers on their sides. (Nordic House and Christians Church, Klaksvik)

Denne løsningen kjentes best ut for meg som musiker. Hørte elektronikken og de andre i ensemblet best.

 

Translation.

This solution felt best to me as a musician. Heard the electronics and the others in the ensemble best.

  1. Ensemble located on stage with front speakers behind. (Copenhagen)

Også ok.

 

Translation.

Also ok.

 

  • Can you describe your personal experience during the concert performance?

Spennende klanger. Litt rart å gi fra seg “kontrollen” over en klang eller en effekt, siden den blir bearbeidet og «fordreid».

 

Translation

Exciting sounds. A little odd to give “control” over a sound or effect since it is being processed and “distorted”.

 

 

Appendix G

Stage setup

Stage setup during performances in Harpa, Reykjavik, Nordic House, Torshavn, and Schæffergården Copenhagen.

 

 

[1]Open Sound Control (OSC) is a communication protocol for networking computers, sound, and other multimedia devices.