Introduction

Narrowed definition of mixed music

In framing my analysis of this project, I have chosen to use the term “mixed music,” which comes from the French “musique mixte” and can be “thought of as being the combination of an acoustic instrument in the presence of electroacoustics.”It is, therefore, a “crossing point between sound-based and note-based music, inheriting the possibilities and problems of both sound worlds” (Lacroix, 2018).In my work on the ConDiS project and in this discussion of it, I will use a narrow understanding of the term and focus on mixed music as a type of semi- and through-composed concert music that incorporates elements of both art music and acousmatic music.

To better illustrate the difference between classic concert performance and classic mixed music performance I turn to Oliver La Rosa’s definition of composition and performance as being “traditionally conceived as a uni-directional, linear, sequential communication process. The composer gives the music to the performer, and the performer plays it for an audience. Two media are used to enable this process: scores and musical instruments” (Oliver La Rosa, 2011). He calls this the composer-performer-listener model. (Fig.1)

Figure 1. The traditional uni-directional communication process of classical concert performance.

In mixed music[1]this “uni-directional” communication is disturbed to an extent since the addition of live signal processing creates an interaction between the performer and the computer. The composition process has therefore changed into one in which the composer not only has to compose the score for the performer, but also the “score” for the computer. Additionally, the model has changed since the computer can “listen” to the performer and respond in real time to his performance. The responding process, or the electronic score (sonic response), is decided (composed) by the composer during the compositional stage. Various methods can be employed to have the computer “listen” and trigger the sonic response, with the most commonly used ones being pitch following, amplitude following, spectral analyzes, and zero crossing.

Figure 2 shows the “traditional” mixed music communication process.

Figure 2. The disturbed uni-directional process of mixed music concert performance.

With the addition of the ConDiS system, a conductor is placed between the composer and the performer(s)/computer in the communication process, as shown in Figure 3.

Figure 3. The mixed music communication process including a conductor.

The insertion of a “new member,” the conductor, between the composer and the performer(s)/computer in this model, adds new dimensions to the communication process. In addition to the responsibilities attached to the classic conducting job, namely interpreting the written score and delivering it to the performers, the conductor now has to interpret the “computer” score as well. This opens up a floodgate of new musical interaction, or what I prefer to call musical interplay. This extended role of the conductor provides conductors new means of expression beyond those they would usually have. Using the ConDiS system the conductor can exercise actual sonic influence by controlling the digital signal processing of the performing instruments. This is in contrast to the “suggested” sonic influence, which is produced by the conductor prompting and spurring action from an ensemble that is actually producing the sound and may or may not follow the conductor’s instinct. Depending on the compositional decisions made, the composer can become a co-conductor, or the conductor a co-composer.

Hence even in pursuing the aim of the ConDiS project, to create a conducting system to use in the performance of through-composed mixed music, the full exploitation of some of these new spaces that have been realized is constrained, and by no means have the boundaries been fully explored. There is an opportunity and a need for further research, additional software and hardware development, and additional experiments with conductors and different styles of compositions and performances.

In Kuuki no Sukima, caution is exercised in harnessing the full possibilities of the ConDiS system, with the main focus limited to the traditional function of conducting, interpreting a written score and controlling volume, sonic balance, tempo, and synchronization. This restriction was chosen mainly due to the fact that a through-composed composition does not require other external conducting methods. Furthermore, the fact that this new expanded conducting capability is still very exotic to conductors recommends a more measured adoption. They are not used to using external elements, and “even having to use a baton has [a] sudden amount of limitation”(Halldis Rønning in an interview). Consequently, I felt I needed to be careful and somewhat conservative in my approach.

Figure 4 illustrates the process model used for the performance of Kuuki no Sukima. The Composer writes a Score for the Conductor to interpret to the performers who react by making instrumental sounds via their written instrumental parts. At the same time, the Composer writes a timeline-based score in the Digital Audio Workstation (DAW)[2], where the entirety of the digital sound process is precisely written to accord with the instrumental score. The Score and the DAW score interact since the Score includes graphical information on the digital sound process and other necessary information such as synchronization markers, volume, and tempo.

Figure 4. The composer-conductor-performer/computer-listener model used for “Kuuki no Sukima”

Figure 5 shows the processes with reference to Oliver La Rosa’s composer-performer-listener model. A third activity category has been added between the categories I prefer to call, Human and Object-oriented activities: Digital activity. The Digital Audio Workstation is situated after the Conductor to emphasize the role of the conductor as the controller, even though the DAW is also a part of the composer’s compositional process. Hence the two arrows from the composer to the DAW in the diagram. Similarly, the conductor controls the performer, who reacts to the conductor’s gestures by playing the written compositional part, making sound via his instrument.

 The instrumental sound then simultaneously travels directly out to the audience and in real time via the signal processing of the DAW. I have chosen to refer to this process as an electronic extension of the instrumental sound.

Figure 5. The mixed music model of Kuuki no Sukima showing the process as three forms of activity: Human, Digital and Object-oriented.

Lack of expressive interplay (Synchronization)

During classic acoustic performances, there is an interplay between the written score and the performers as well as between the performers themselves. The performers read the composed score and add expressions, emotion, and dynamics using their musical training, experience, and musicality. In mixed music, electronic sounds are added to the instruments performing the composition and in doing so the interplay gets somewhat lost. Regardless of whether the electronic sounds are played from a pre-recorded audio file or an automated timeline of a digital audio workstation (DAW), they lack comparable expression—they lack the human touch. What is ultimately missing is synchronization and the crucial musical interplay between the instrumental and the electronic.

Often the solution is to write the composition in either strict metric or free semi-improvisational style. Strict metric style means the performers would have to follow a given tempo quite closely, sometimes wearing headphones playing a click-track in their ears. In the free semi-improvisational style synchronization becomes less critical with performers playing freely in and out of synchrony with the electronics. Writing through-composed music for mixed music media is a challenge especially when writing for larger ensembles. In these cases, synchronization between the acoustic instruments and the electronics becomes critical. This is what spurred the creation of ConDiS – Conducting Digital System.

The lack of expressive interplay has been a continual challenge for contemporary music since the first appearance of electronic music. From the 1980s on, various attempts have been made, including explorative efforts by Michel Waisvisz (Waisvisz, 1985)Philippe Manoury and Miller Puckette (May, 1999), Morton Subotnick and Mark Coniglio (Rowe, 1992), and Tod Machover and Teresa Marrin Nakra (Paradiso, 1999), among others. In these attempts the performer is the source of expressive timing and variation. Less explored is how to allow the conductor to shape the music, as she would do with purely acoustic music.

What turns a written composition into music is the musician’s interpretation of the written musical score. It is what is written in-between: the interplay between what is written and what is performed. It has everything to do with the interpretation of music in time and space; hence the need for ConDiS, which allows the conductor to expressively control and synchronize the music produced by the performing instruments and the electronic component of through-composed music.

Compositional Interest

My main compositional interest lies in the field of through-composed contemporary music using live electronic extension of the instrumental sound. Unfortunately, until recently, this style of through-composed mixed music in a larger format (ensemble/orchestra) was not technologically feasible given the precision such a composition method requires. Through-composed music needs this preciseness, it needs to be interpreted in a similar way from one performance to another. This description should not be taken literally, however, as a certain degree of freedom of musical interpretation can certainly be incorporated, as is evident in my own compositional methods.

Faced with a choice between the strict metric or semi-improvised style, the semi-improvised style has been more appealing to me due to its flow and fusion. I am drawn to its ability to flow freely according to the musical interpretation of the performer(s) and the accompanying electronic sounds. This captivation can be traced through my mixed music compositions: the Sononymussequence for different solo instruments to Circles I[3]for three instruments, Hljóðs biðekfor voices[4]and Hljóðs bið ekfor five instruments[5]. In these compositions various experiments are performed, both compositionally and technologically, dedicated to the search for a way to synchronize the written score and electronics. I have found pitch tracking to be the most successful.[6]This is especially evident in my Sononymussequence for solo instruments and computer-generated sounds, which is not entirely surprising since there is only one instrument interacting with the electronics. Including more instruments complicates the pitch tracking due to sound bleeding between the microphones.

In my composition Circles I, I work around the problem by integrating the bleeding into the construction of the composition. Circles Iis written for three instruments that circle between eight different electronic effects—the circling is based on a set of pitches, which trigger jumps from one setup to another. So bleeding between the microphones causes disturbances which then alter the adaptive effect setup controlled by the pitch parameter. Thus the varying bleeding in the microphones causes variation in the electronic effects. The pitch-tracking disturbances and their consequences are therefore part of the composition. The result is a composition that varies significantly between performances.

In order to reduce these variations and compose music with more precision, as required of through-composed music, I needed to create the ConDiS system; the possibilities bestowed by this system are exemplified in the composition Kuki no Sukima.

Circles I – Defun Ensemble in Helsinki

 

Content and structure

This ConDiS – Conducting Digital System report is divided into seven main chapters in which I reflect on distinct aspects of the project. This traces the process of creation, starting with an unconscious need all the way to the development of a fully-fledged system used for the performance of through-composed mixed music.

I approach the role of the conductor in the same way, as her role is one of the core focuses of this research. In my conversations with people outside of the field of music, it became clear to me that the majority has little knowledge of the role of the conductor, such as how she uses gestures to control rhythm and volume, among other things. I have therefore devoted space in this paper to explain the importance of the conductor, given this figure’s central place in the research project. This will improve the reader’s understanding of how the role of the conductor is extended through the ConDiS conducting glove, ConGlove.

The third and final chapter of this section gives a brief overview of the technical development of the software and hardware related to my research. I conclude the chapter with an explanation of my decisions regarding the structure of the ConDiS system.

The chapter entitled “Technological Development” reviews a variety of aspects related to the choice of hardware and software. This includes the presentation of results of experimentation with different arm gestures, and the chapter ends with video examples of the practical use of these gestures to control output.

“The Research Process” chronicles the project’s entire process of artistic evaluation, focusing on considerations and needs from an artistic, as opposed to technological, perspective. Particular emphasis is placed on the importance of researching and developing score instructions for the electronics, since this is one of the most important foundational aspects of unified interplay in mixed music. To this end, I analyze three experimental compositions I wrote with the purpose of trying out various graphics specifically in mind.

Chapter 6 “Performance Reflections” refers to reflections based on:

• Performance preparation;
• Performance analyses;
• Interview with conductor Halldis Rønning;
• Questionnaire sent to members of Trondheim Sinfonietta; and
• Conversations with audiences.

These reflections are made through an analysis of the written musical score in conjunction with audio and video recordings made during the performances. I reflect on the composition as well as the performance, emphasizing both positive and negative aspects. The interview with conductor Halldis Rønning was conducted via Skype shortly after the Nordic Tour and was guided by a questionnaire I sent her in advance. I also used an email questionnaire to survey the members of the Trondheim Sinfonietta who had performed during the Nordic Tour. The email was sent shortly after the tour, but the response rate was rather low, with only three completing the survey. Despite this low response rate, I consider their answers consistent enough for me to draw valid conclusions from the collected information.

The final section of the report— “Conclusion and Future Work”—summarizes the overall results of and experiences during the ConDiS artistic research project. In this I place particular emphasis on the artistic gain realized through “learning by doing.” Finally, I explore concepts for the future development and practical expansion of the project.

The “Epilogue” is based on my reflections following the closing concert of the project. The concert was not originally conceived as an object for reflection, but rather as an endpoint for the project and a result of the analyses themselves. However, I realized that providing thoughts on the closing concert would benefit the reader by providing a better understanding of how the research had progressed and an impression of the last stage of the project in this iteration.

Research and context

The research process can be divided into five main categories in chronological order.

1. Technology/Technical– Development of the device itself, i.e. software and hardware. Choosing which software to use and then completing basic programming in Max/MSP constituted an important part of the research. Using a do-it-yourself method meant that I felt more direct contact with the execution of the project. I therefore made do without any external assistance, with just a few exceptions, in cases where functional changes were needed, additional programming was called for.
2. Graphic Notation and Instructions– Focusing on developing symbols for the electronic sound processing and related instructions for conducting.
3. Experimenting– Using experimental compositions to test the interface to and the functions of the hardware, the conducting glove “ConGlove.” Compositions in the form of solo études provided the opportunity for electronic and aesthetic troubleshooting. I collaborated with individual performers to work out various extended instrumental techniques.
4. Composition– Composing a new work Kuuki no Sukima, which was purpose-written to demonstrate the advantage of the ConDiS system for through-composed mixed music performances.
5. Performance– Collaboration among the composer, the conductor, and performers. Rehearsals and concerts.
6. Conclusion and future work – New compositions to further explore the potential of ConDiS and future collaboration between composer and conductor, other conductors, students, ensembles, and developers. Future developments and improvements to the ConDiS system are discussed.

Certain questions accompanied the ConDiS research project throughout. These were kept foremost in mind during the artistic research process:

1. Why– Why do I need the ConDiS system? What do I want from the ConDiS system?
2. Process– Is there an artistic advantage to this research process?
3. Composition– Will the ConDiS system produce a different style of composition?
4. Conductor– Does the use of ConDiS maneuver the conductor into a different and more creative role?
5. Performance– Will the extended role of the conductor—in which she actively functions as a co-composer or performing member of the ensemble—change our expectations of the concert performance as traditionally understood?

[1]I am referring to a specific form of mixed music where live processing is used. This discussion therefore excludes the form where the electronic component is fixed.

[2]An electronic device or application for recording, editing and producing audio. The abbreviation will be used throughout this paper.

[3]Circles I on YouTube, Circles I in Helsinki

[4]Hljóðs bið ek for voices on YouTube, Silence I Ask for Voices

[5]Hljóðs bið ek for five instruments on YouTube, Silence I Ask for five instruments

[6]Pitch tracking means the process of analyzing pitch in an audio signal.