Capturing, Tracing, Transferring the aural flux Convergence, interactivity, in situ composition and emergent-instruments in […]S[…]S[…]S[…]

Mauricio Arturo Meza Ruiz – [email protected]
  University of Reims Champagne-Ardenne/Conservatory of Reims/Césaré-CNCM

In a good part of the interactive devices conforming the project […]S[…]S[…]S[…]1, a series on which I’ve been working since 2005, a metaphor of aural collective memory is created on the first part of the process. Subsequently, this same process generates the conditions for a performance to take place, in which the action unfolds as a new endogenous process that I call in situ composition. When the convergence of this processes reaches its climax, the performative situation is enlarged towards an event that is much like an electroacoustic mixed music concert. In this one, the audience can experience the manifestations of an entity whose qualities and characteristics tend to determine a musical and conceptual object on which I’ve been working since 2012, and that I call: emergent-instrument. The theoretical frame at the foundations of this body of work relies on what I consider as a compositional operative regime based in the interaction between three core notions : capture, trace and transfer. This regime on itself is completely dependent of an interactive relationship with sound that I conceive as a dialogical relation with aural matter, that is: performance with sound, mediated by digital technologies. In this article, I will take-on the subject of in situ composition, the methodology it implies and some of it’s outputs by means of introducing the notions of capture, trace and transfer, in the context of a general description of one of my interactive devices, Doppelgänger_Oblivion: Timeline (2012). Starting by underlining the aspects that make of it a space and a time of interactivity and convergence, my exposé will point at some prospective aspects of the emergent-instrument while at the same time, offer a description of some of the functions implemented in the software used in my interactive devices for the purpose of in situ composition.

Figure 1

Figure 1. Doppelgänger_Oblivion: Timeline (2012). A fragment of the visual flow generated in real-time in the installation.

Introduction

The hint-corner stone of a body of work

All of the instrumental composition works of my first period2 are hand-made pieces that I wrote, each on its own time, in one stroke from beginning to end, allowing very scarce intervention of technology in the process. Although, as a part of my creative engagement, I have always been actively questioning the possibilities and artistic outputs of technology, willing to find my way into the digital realm and how to achieve its formal integration into my compositional process, which concentrates primarily in the composition of chamber music. As a consequence, I undertook into the creation of another body of work, parallel to my instrumental composition output, focusing primarily on the exploration of the possibilities of interactivity, as it is conceived in the digital domain. From the tension produced by this double bind frame of work, raised a question that became the hint-corner stone of the reflections nurturing the work I have been engaged in lately: what is a musical culture and how musical imagination appears and individuates itself ?3 Assuming that every musical culture emerges and takes-shape throughout a historical and temporal complex process, in which a large variety of terms, conditions, factors and entities enter into systemic interaction, one of the purposes of this project is to create a body of work reflective on the poetics of sowing-and-harvesting. Such a body of work shall be constituted by different types of artistic output, going from instrumental and mixed compositions, to sound installations, interactive multimedia devices and furthermore, a radio-opera.

A protocol for […]S[…]S[…]S[…]

Moreover, what interested me about digital technologies, just as much as interactivity, is the broad range of possibilities they offer in terms of the creation and formalisation of convergences4, either between the materials of art themselves, as well as between the diversity of aesthetic operative behaviours5 art itself implies. As a consequence, the first challenge and the first task to be carried out in this context, is the constitution of an artistic research and experimentation protocol, following the idea of ​​an aurallity emerging from the interactivity6 among an ensemble of inter-linked entities, in the centre of which is a dialogical relationship with sound7. From the interaction with sound and visual flows generated in real time, and making usage of generative-evolutionary large temporal-arc processes, as well as the constitution of databases, such a protocol should as well be comprised as a poetic play ground for the observation of different kinds of correlations between aural perception, the proprioceptive system, natural and artificial memory, and the cultural and natural environments. In order to explore some of the many possibilities suggested by these premises, I have been engaged in the production of a large-scale, long-term, and interdisciplinary body of work I’ve entitled […]S[…]S[…]S[…] (a.k.a Seed / Simiente / Semance, a.k.a 3S, a.k.a Tres-es series, a.k.a The Trip-less series). Particularly, since 2011, I’ve had the opportunity to create an ensemble of interactive multimedia devices8 all of which follow, partially or totally, the structure provided by the same protocol.

A compositional operative regime for in situ composition

In a good part of the interactive devices conforming the […]S[…]S[…]S[…] project, a series on which I’ve been working since 2005, a metaphor of aural collective memory is created on the first part of the process. Subsequently, this same process generates the conditions for a performance to take place, in which the action unfolds as a new endogenous process that I call in situ composition. When the convergence of this processes reaches its climax, the performative situation is enlarged towards an event that is much like an electroacoustic mixed-music concert in which one can experience the manifestations of an entity whose qualities and characteristics tend to determine a musical and conceptual object that I have been calling since 2012, emergent-instrument9. The theoretical frame at the foundations of this body of work relies on what I consider as a compositional operative regime based on the interaction among three core notions, which are: capture, trace and transfer. This operative regime on itself is completely dependent of a collective and individual interactive relationship with sound that I conceive as a dialogical relationship with aural matter, that is: live performance with sound, mediated by digital technologies. In this article, I will take-on the subject of in situ composition, the methodology it implies and some of it’s outputs by introducing the notions of capture, trace and transfer in the context of a general description of one of my interactive devices, Doppelgänger_Oblivion: Timeline (Dieburg, 2012)10. Starting by underlining the aspects that make of it a space and a time of interactivity and convergence, my exposé will point at some prospective aspects of the emergent-instrument while at the same time offer a description of some of the functions implemented in the software used in my interactive devices for the purpose of in situ composition.

Converging processes in Doppelgänger_Oblivion: Timeline

Doppelgänger_Oblivion: Timeline, a title that could be translated as “abolition of the double in the line-of-time”, is a device11 that is first presented as an interactive multimedia installation, then, in the course of a circular process12, it progressively transforms itself into a space in which an ensemble of entities, situations and activities get involved, with their respective spaces and regulators, in a relationship towards the generation and the evolution of the symbolic content, and audio and visual output of the system itself. Thus, an interactive multimedia installation, a composition workshop and a space for musical performance converge in the same space and time of exposition of the device. In the last instances of the process, as a result of the recurrence of this inter-activated convergences, emerges a new and autonomous entity, whose structural, material and behavioural characteristics, and whose aesthesic qualities of sound, visual, spatial and symbolic manifestations suggest, in the manner of a musical instrument, a framework of reference for the composition of a particular kind of electroacoustic mixed music, where an interpreter interacts with the composed and emergent qualities of such an entity. Figure 2 illustrates the convergence of modes, situations and processes as they occur in the time of exposition of the device.

Figure 2

Figure 2: Visualisation of one capture-trace-transfer cycle.

First, by tracking the behaviours of the visitors, the installation mode enables a process of capture, which in turn leads to the creation of a database — a metaphor of collective aural memory. Afterwards, this same process generates the conditions for the emergence of the in situ composition process. Convergent with the first one, this process engages the composer as a performer within the installation. As it allows the composer to explore the database and to use its content for creative purposes, this phase in turn, is at the crossroads of the capture and transfer operative realms. When the convergence of this iterative situations and processes reaches its climax, the performance is enlarged towards a particular event that invites a musician to perform a score in an interactive situation involving the system as an instrument. Much like an electroacoustic mixed music concert, this phase allows the visitors to experience the manifestations of the emergent-instrument, an entity whose qualities and characteristics are endogenic and endemic in relation to the ecosystem represented by the device.

Capture process: Interactive multimedia installation mode

Interactive sound space

In the interactive multimedia installation mode of Doppelgänger_Oblivion: Timeline13 [Figure 3], a continuous flow of sound and light inhabits the space. The visitors are invited to wander in, while in the course of their visit, they may discover, or not, that their walk influences the behaviour of the flow of sound and light. Sound is generated in real-time by means of additive synthesis, controlled by a programmed looped structure that goes progressively from the projection of a single sinusoidal wave, all the way up into a wall of sound14, passing through successive stages, from the constitution of a harmonic spectrum, to an inharmonic one, until it reaches a state of relatively high saturation and then, back forward progressively into the projection of a single sinusoidal wave. On the other hand, the sound generated by additive synthesis is transformed in real-time throughout its evolution, with a combination of treatments like distortion, delay, reverb and granulation. This allows for the walk of the visitors to influence the resulting sound on its behaviour and its evolution, both at the level of its macro-temporal and micro-temporal articulations, resulting in a modulation of the overall behaviour of the additive synthesis cycle, from one instantiation to another of the capture-trace-transfer cycle. As for the light flux [Figure 1], it is generated by means of a video projection of synthetic images, which in turn evolve in correlation with the overall sound process15.

Figure 3

Figure 3: Original sketch of Doppelgänger_Oblivion: Timeline. On the 2012 version, the Kinect sensor was replaced by a camera.

From capture to timeline and generative aural and visual outputs

When a visitor enters the installation, a video camera detects his / her position and then, by means of a program written with Max16, the speed and the direction of his or her walking are calculated. With this information captured by the system, a database is created, assigning to each visitor a data-structure17. There after, from a specific data-structure and according to its pertinence in relation to a set of rules, the program generates a timeline corresponding to the behaviour of a particular visitor. This timeline is then on the one hand, stored in memory for a later use while on the other hand, it is compared and filtered in real-time by the program with respect to a previously written master-timeline. From this operation of analysis, filtering and concatenation, the software determines the evolutionary course of the multiple parameters18 that come into play in the generation of the sound and visual flow projected by the installation [Figure 4].

Figure 4

Figure 4: Real-time parameters programming interface for Doppelgänger_Oblivion: Timeline. The resonators where not used in the 2012 version.

Engaging the visitor towards attentive listening

In this first phase of the process, as the visitors are not allowed to learn that their movements are being recorded by the system, the purpose to record their behaviours in order to stock them and create a data base is therefore not intended to be part of the visitors first aesthetic appreciation of Doppelgänger_Oblivion: Timeline. Moreover, this implies that the interactive installation mode of the device has to stand on its own as a unique, and necessarily interesting, aesthetic experience for each visitor19, appealing to his or her engagement into the transformation of the aural and visual environment. Thus, at this stage, the visitor is expected to assume a critical position towards what he or she hears and sees, and from that point on, being able to express, through her / his ambulation, an index of sensitive preferences. The protocol at the foundations of this development implies that these sensitive preferences are, a priori, captured by the system, that is, the time spent in the installation by the visitors, as well as the shape resulting form his or her overall displacement, provide an index of intensities that can be correlated to a coefficient of attentive listening.

Trace process: Ecosystemic performance mode

From awareness to participation

The protocol implies the assumption, by the visitors, of a greater responsibility for their actions, as they become more aware of what the interactive situation has to offer. Hence, their status evolves from that of occasional, casual, lambda visitors to that of performers, thus taking part in the overall creative process proposed by the ecosystemic performance context offered by the device. From here, the performance mode of Doppelgänger_Oblivion allows to consider each of the participants, including the machines, as potential performers that progressively contribute, through their interactions, to the establishment of an ensemble of conditions that lead to the emergence of an artificial musical culture, proper to Doppelgänger_Oblivion as an ecosystem. Figure 5 illustrates the relationship amidst the performers in the ecosystem.

Figure 5

Figure 5: Functions of performers in the interactive ecosystem20.

Building a database, an in situ composition task

As foretold, after some time running as an installation, the system builds a database from the information provided by the visitors through their behaviours [See figures 6 and 7]. This allows Doppelgänger_Oblivion to initiate a new process in which the installation becomes the background for a performative situation. In this one, a composer, or a sound artist, is invited to perform the writing of a composition in the situation offered by the installation, that is, to intervene assuming her / his own role as a composer in a performance within the same space-time of the installation. So, as the installation mode is still running21, the functions of an interactive multimedia installation converge with those of a composition workshop22 that allows, on its own means and terms, to create a composition from the different materials generated within the same interactive installation. Thus, the data structures conforming the database should be considered as essential factors in the sonority of the instrument-in-process-of-emergency, however, the composer-in-the-situation has authority to include or discard, with respect to his or her in situ composition project, the sonority implied by these blocks of information, that is, one can choose, for example, to use them as pre-written whole sections to be inserted in the score and / or using them by giving preference to other structural or aesthesic of their qualities and potentials, some of which the composer should ideally discover during the in situ composition phase.

Figure 6

Figure 6: Visualisation of an excerpt of the database of Doppelgänger_Oblivion: Timeline.

Figure 7

Figure 7: An excerpt of data structure corresponding to “user211”. The date and the time, attributed automatically by the software, indicate the beginning of the walk in the installation.

In an out of the system: towards a dialogue between more than two

worlds

Ideally, as suggested before, the output of such a performance of in situ composition should be in resonance with the interactive, symbolic, aural and visual qualities of the ecosystem the device represents itself. Thus, the composer encounters with the challenge to find a way to preserve the stylistic elements of his or her own compositional language, while at the same time, sustain the aesthetic qualities prefigured by the system. Nonetheless, the protocol not only allows, but demands the projection of the device’s resonance far beyond the frontiers of the ecosystem. This is the case, for example, of my second string quartet, a forth coming piece on which I have been working since 2015 and where, in order to generate the materials for this particular work, I make usage of the Doppelgänger_Oblivion: Timeline, and the [L’emprise de Panurge]:::[Du feu de Toungouska], as well as the Un theatre des opérations23 data bases. [See figure 10].

Composition tool for retrieving the “future-past”

In order to make use of the information generated by the system for in situ compositional intent, the same software implemented to allow Doppelgänger_Oblivion to run as an interactive installation-performance, allows access to the data captured during the first phase of the process, opening the way for the application of different methods in terms of how to interrogate the database for creative purposes. Recall and retrieve, graphic generation, reverse play, as well as temporal compression and dilatation, figure among the functions hereafter described and already implemented in the software. Still, this functions should be understood not as common-core composition digital tools-of-the-trade that one can find here and there, among the many existant software developments, but rather as endogenous and emergent qualities, bearing potential functionalities, that the system itself brings about, all the way from the process of its conception, up until its mise en route, realisation and current development.

Recall / retrieve function

As every trace left by the visitors in the system is correlated to the dynamic and parametric control of real-time transformation of the audio-visual projection, this function affords with the possibility of seeing and listening to a particular state in which the aural and visual flow was behaving at some specific instant of its evolution. By allowing a perceptual analysis of the aural and visual properties of a particular instance in the history of the ecosystem, that is, allowing to re-run the parameters involved in the transformation of the flow of sound and light between a moment ‘a’ and a moment ‘b’ of its gradual development, this function provides the composer with the means to evaluate and classify, from a sensitive perspective, the traces left in by the visitors-performers, in the memory of the ecosystem.

Graphic generation

By implementing the possibility to render a graphic representation of the ambulations of the visitors in a two-dimensional rectangular space, where axis ‘x’ represents the duration of the movement and axis ‘y’ corresponds to its direction, it becomes possible in this way to operate a transfer of the spatial and temporal indices generated by the displacement of the body in the installation towards other spatial and gestural realities, such as the one represented in the instrumental domain. Figure 8 shows the graphic visualisation generated from the trace left by visitor “user211” ; the software allows for a modulation of the rate of graphic-visual resolution.

In the first instantiation of Doppelgänger_Oblivion, a score for solo trombone and emergent-instrument24 was written and performed 25. In this one, the lines that describe the displacement of the visitors-performers are interpreted by the trombonist either as modulations to be carried out with the pressure of the lips, or as glissandos to be played with the instrument’s slide bar. Other articulations were inserted in the score, all of them related to the technical possibilities of the trombone [Figure 9]. As for my second string quartet, on its current state of development, the data corresponding to the displacement of the body of the visitors-performers is being transferred to the movement of the left hand across the fingerboard [Figure 10].

Figure 8

Figure 8: Graphic visualisation of the trajectory of “user211” at two different rates of resolution.

Figure 9

Figure 9: Excerpts of the score for trombone and Doppelgänger_Oblivion: Timeline as emergent-instrument. Composed in situ in 2012.

Figure 10

Figure 10: Excerpt of the draft of Esquisse pour une tracéologie, for string quartet (work in progress started in 2015). This excerpt uses the database of Doppelgänger_Oblivion: Timeline. The rhythm part was generated from an older database build with a2m.26

Reverse play, and temporal compression and dilatation

Temporal compression and dilatation, as well as reverse-play functions are implemented in order to allow adaptation to the constraints implied by the usage of traditional instruments and by the same token, afford with a more tangible flexibility of articulation between the digital and the analogical domains of the ecosystem, as is the case, for example, when dealing with breathing with the trombone. In Doppelgänger_Oblivion: Timeline, each of the aural and visual frames corresponding to the data structures chosen by the composer-performer for the trombone part were projected alongside the instrumental execution, with some of them in reverse play and / or compressed mode. As for the temporality proper to each memory-trajectory data structure chosen for the performance, operations of temporal compression and / or temporal dilation were applied, in such a way that the duration of each projected trace is relative to that of the instrumental gesture. Some other traces were projected in their original form and duration, intercalated with the parts of the trombone.

Transferring the flux: a projection towards the embodiment of the emergent-instrument

As it has been stated before, in the last stage of the process, the convergence between the modalities of installation and performance entails the possibility of the integration of a musician as an interpreter, so that the device manifests, on the one hand, the qualities of an aural projection space suited for the interpretation of the same score generated during the preceding stages, and on the other hand, the qualities of an electronic instrument whose behaviour and sonic, visual and spatial attributes are intimately linked, both in an ontogenetic and phylogenetic way, to the whole generative, transformational and evolutive process of the ecosystem. By train of consequence this implies, ideally, the generation of new dialogical conditions within every cycle of capture-trace-transfer processes, that which in turn results, with every recurrence, in the composition and the interpretation of a piece in which the instrumentalist inhabits, in the course of the performance of the score, an aural space that in turn is, an integral and emergent part of the system.

Towards music composition as collectively programming the individual

The type of electroacoustic mixed music emerging from the process described here implies a particular approach to composition. In situ composition, in this context, needs first of all to be established from the perspective of a formal approach to digital convergence and interactivity, this in turn, traduces into the conception of a dynamic space in which a capture of information is performed. The data captured here is that of the interactivity happening in the device as an installation. My assumption is that this kind of interactivity, proper to the digital realm, should be conceived as the pursuit of a dialogical relationship with sound, such a relationship being articulated by the dynamism of the body and mediated by digital code. The next step is to interrogate the outputs of the capture processes in order to determine what might have a function of material for the means of composition. Here, the composer-performer, in the manner of an archeologist observes, during the time of convergence between the installation and performance modes, some of the data structures generated by the system, and thus determines their relevance for a composition project to be made within the context offered by the device itself: a performative situation, in addition to a dialogical one. Finally, a transfer of the structures captured in the domain of interactivity to the domain of musical performance is operated. Furthermore, the idea of an instrumental entity emerging from the convergence taking place in this ecosystemic environment implies the inclusion of the aural and visual materials generated in the installation-performance phase as part of the in situ composition and consequently, in situ interpretation projects. This in turn allows me to touch on the subject of the status of the digital trace, as it is comprised in this context, by proposing a preliminary two-pronged definition of it. Firstly, the trace should be comprised as code emerging from a composed and mediated aural interactive situation — a programmed dialogical relationship with digital sound — that has transferable qualities and properties such as an index given by a date of beginning, a shape, a temporality of its own, and correlativity to historical sound parameters and visual and symbolic outputs. And secondly, as being the point of convergence, both at a structural and an operational level, among the capture and transfer realms. Hence, the digital trace, having a particular status concerning the articulation of the different modes of activities and participation in the ecosystem, the whole process implied by the device moves towards collectively writing the program of the emergent-instrument.

Conclusion

The protocol at the basis of this development can be synthesised by the operative conjunction: capture-trace-transfer, where capture is to be understood as the operation of exchange, recollection and symbiotic inter-conditioning ; trace-tracing is the operation of rendering what is pertinent for the purposes of in situ composition ; transfer is the operation of re-enacting what has been given in the course of capture and trace processes or, allowing myself an expression inspired by the philosophy of Jacques Derrida: re-actualising the system by lending a presence to the ghosts of the visitors. In the last instance of the process described here, in situ interpretation, as the last stage of the operation of transfer, concerns the musician, both as performer and interpreter, as well as the machine, as it mediates the whole process within itself, suggesting the involvement of an agent-code whose status of steward of the interactivity, and withholding inherent aesthesic qualities, just as much as operative potentialities, suggest in turn a certain amount of autonomy. Moreover, in a forthcoming development, the system should be able to regulate itself in order to stabilise its audio-aesthetic output, and hence be able to render an aural proposition of its own. Capture, trace, transfer and moreover, emergency and individuation, as well as digital convergence, all of it in the context of my artistic project, suggest a compositional approach on the convertibility bearing among structures and operations, that is, an allagmatic27 approach to music creation. As the core question at the foundations of this project is that of the emergence of music as culture and imaginary, the file rouge of the collective performance taking place in the installation is that of the birth of music and the dramaturgy that such an event could imply. All of what should be deepen in the subsequent developments of the “Trip less: abolition of the double in the line-of-time”.

Bibliography

Agamben, Giorgio. Qu’est ce qu’un dispositif. Paris: Éditions Payot Rivages, 2014.

Ashby, W. Ross. An introduction to cybernetics. Londres: Chapman & Hall Ltd., 1957.

Barbaud, Pierre. Initiation à la composition musicale automatique. Paris: Dunod, 1965.

Barkati, Karim : “Notice de a2m-009c”, December 2008 (online): http://karim.barkati.online.fr/Universite/Doctorat/Logiciels/ApplicationsMacOS/a2m/ (visited 23 March 2017).

Boissiere, Jean-Louis. La relation comme forme. L’interactivité en art. Genève: Éditions Mamco, 2004.

Bossis, Bruno. “Ecriture instrumentale, écriture de l’instrument.” In Composer au xxie siècle. Pratiques, philosophies, langages et analyses. Edited by Sophie Stévance, 119-135. Paris: Vrin, 2010.

Criton, Pascale, and Jean-Marc Chouvel, ed. Gilles Deleuze. La pansée-musique. Paris: CDMC, 2015.

Cowell, Henry. New musical resources. New York: Cambridge University Press, 1996, (first edition 1930).

Chouvel, Jean-Marc. Analyse musicale: Sémiologie et cognition des formes temporelles. Paris: L’Harmattan, 2006.

Chouvel, Jean-Marc, and Xavier Hascher, ed. Esthétique et Cognition. Paris: Publications de la Sorbonne, 2013.

Couchot, Edmond, and Norbert Hilare. L’art numérique. Paris: Flammarion, 2003.

Couchot, Edmond. La nature de l’art. Ce que les sciences cognitives nous révèlent sur le plaisir esthétique. Paris: Hermann Éditeurs, 2012.

Couchot, Edmond. “Médias et immédias.” In Art et Communication. Edited by Robert Allezaud, 101-106. Paris: Éditions Osiris,1986 http://www.multimedialab.be/doc/citations/edmond_couchot_medias.pdf (visited 23 March 2017)

Deleuze, Gilles, and Felix Guatari. Capitalisme et Schizophrenie. Tome 2: Mille plateaux. Paris: Editions de Minuit, 1980.

Derrida, Jacques. De la grammatologie. Paris: Editions de Minuit, 1967.

Derrida, Jacques. Trace et archive, image et art. Bry-sur-Marne: INA Éditions, 2014.

Di Scipio, Agostino. “ «Sound is the interface»: from interactive to ecosystemic signal processing.” Organised Sound Vol 8 No. 3 (2003): 269-277.

Estrada, Julio, “Le continuum in musique: structure et ouvertures in composition, ses derivations esthétiques.” In Ästhetik und Komposition. Aktualität der zur Darmstädter Ferienkursearbeit. Edited by Borio, G. and Mosch, U., 50-65. Mainz: Schott, 1994.

Gardiner, Michael E. “Le défi dialogique de Bakhtine aux sciences sociales.” Slavica Occitania 25 (2007): 67-87.

Leman, Marc. Embodied Music Cognition and Mediation Technology. Boston, Massachusetts: MIT Press, 2008.

Merleau-Ponty, Maurice. Phénoménologie de la perception. Paris: Éditions Gallimard, 1945.

Maturana, Humberto, and Francisco Varela. Autonomie et connaissance: Essai sur le vivant. Paris: Editions du Seuil, 1989.

Meza, Mauricio, and Frédéric Dufeu. “Dopelgänger_Oblivion: Timeline. An interactive multimedia installation for collective in situ composition.” Proceedings of the international conference The global composition. Conference on Sound, Media, and the Environment. Dieburg: 2012, pp. 459-461.

Rancière, Jacques, “The Emancipated Spectator” In Artforum (2007): 271–281.

Schaeffer, Jean-Marie. Adieu à l’esthétique. Paris: Presses Universitaires de France, 2000.

Sebbah, François-David, “Traces numériques: plus ou moins de fantôme(s) ?”, In Le sujet digital, edited by Larsonneur, Claire, Regnauld, Arnaud, Cassou-Noguès, Pierre, Touiza, Sara, 114-126. Paris: Les presses du réel, 2015.

Sha, Shin, Wei. Poiesis and Enchantment in Topological Matter. Boston, Massachusetts: MIT Press, 2013.

Simondon, Gilbert. Du mode d’existence des objets techniques. Paris: Éditions Aubier, 1958.

Simondon, Gilbert. L’individu et sa genèse physico-biologique. Paris: Editions Jérôme Millon, 1998.

Stockhausen, Karlheinz. “Wie die Zeit vergeht.” Die Reihe n°3 (1957). Translated to french by Christian Meyer: “Comment passe le temps”, Contrechamps n°9 (1988): 26-65.

Varela, Francisco, and Evan Thompson, and Eleanor Rosch. L’inscription corporelle de l’esprit. Sciences cognitives et expérience humaine. Paris: Seuil, 1993.

Weissman, Joseph, Fractal Ontology blog https://fractalontology.wordpress.com/2007/11/28/a-short-list-of-gilbert-simondons-vocabulary/#more-342 (visited 23 March 2017)

Xenakis, Iannis. Musiques Formelles. Paris: La Revue Musicale n°253-254, Editions Richard-Masse, 1963.

Notes

1 On the summer of 2015, I was honoured by an invitation to participate in the program of the Musica de Agora na Bahia (MAB) festival, in Salvador Bahia, Brazil. After the world premier of one of my works in the context of the festival, I had the opportunity to give a lecture on the subject of my research in composition. This article resumes the content of my lecture, while at the same time it gives an overview on the current state of development of the project I presented on that particular opportunity.

2  From 2000 to 2008, I’ve taken all of the works from 2000 to 2003 out of my catalogue. A second period could be established between 2009 and 2014.

3  An apparently naive question when one does not take in account the prospective context on which it is formulated, where I consider, for the means of poetic, scientific and musicological research, the high speed at which technological development is ongoing, and the actual and forthcoming advances of cognitive science and Artificial Intelligence.

4  A more pragmatic approach in terms of realisation and implementation derives form the choice of this notion, nonetheless, I consider my current approach to formal digital convergence as a step towards hybridisation, a key notion in Edmond Couchot’s theory of digital arts. See: Edmond Cuchot, “Médias et immédias.” In Art et Communication. Edited by Robert Allezaud, 101-106. Paris: Éditions Osiris,1986 http://www.multimedialab.be/doc/citations/edmond_couchot_medias.pdf (visited 23 March 2017)

5  The concept of aesthetic operative behaviour is understood here accordingly to the etiological hypothesis defended by Jean-Marie Schaeffer in Adieu à l’esthétique (2000). Couchot will as well take Schaeffer as a reference for his work La nature de l’art. Ce que les sciences cognitives nous révèlent sur le plaisir esthétique (2012).

6 I take as a reference the theory of cognition proposed by Maturana and Varela (1980), where knowledge emerges from the interactivity of at least two autopoietic entities.

7  The notion of dialogue should be understood accordingly to the dialogic of Mikhaïl Bakhtin: «It is the idea that at the most fundamental level, the universe consists of phenomena constituted by and in the course of reciprocal relations with other forces and entities - and, moreover, that human beings are absolutely involved in these dynamic processes. As a result, Bakhtin emphasises and values perpetual transformation, ambiguity and interaction, counter to modern predilection for order, immutability, symmetry and predictability» Translated by the author form: «C’est l’idée qu’au niveau le plus fondamental, l’univers consiste en phénomènes constitués par et au cours de relations réciproques avec d’autres forces et entités – et de surcroît que les êtres humains sont absolument impliqués dans ces processus dynamiques. En conséquence, Bakhtine souligne et valorise la transformation perpétuelle, l’ambiguïté et l’interaction, à rebours de la prédilection moderne pour l’ordre, l’immuable, la symétrie et la prévisibilité». Gardiner, Michael E. “Le défi dialogique de Bakhtine aux sciences sociales.” In: Slavica Occitania 25, (2007): 67-87.

8 Oxyde: The Whale Song (Paris, 2011), [L’Emprise de Panurge.]:::[Du feu de Toungouska] (Reims, 2015), Semences d’écoute: Un théâtre des opérations (Reims, 2016).

9  In 2012, at the presentation of Doppelgänger_Oblivion: Timeline, at The Global Composition. Conference on Sound, Media, and the Environment, in Dieburg, and at the Open Space, curated by Berno Odo Polzer for the Internationale Ferienkurse für Neue Musik Darmstadt. In 2013, at my lecture on my compositional process, at the Projeto Compositores seminar, at the Universidade Federal do Rio de Janeiro, on the invitation of Pauxy Gentil-Nunes and Pedro Bittencourt. In my lecture entitled «Doppelgänger_Oblivion: Timeline. A multimedia interactive installation-instrument leading to in situ composition», for Semántico, international symposium on generative art, Mexico City, 2014.

10  The first instantiation of the device took place at the international conference The global composition. Conference on Sound, Media, and the Environment, in Dieburg, on 2012.

11  The term “device” is used here as translation of the term dispositif, in french. I take as a reference the work of Giorgio Agamben, Qu’est ce qu’un dispositif?, where he addresses a critic of this notion in the philosophy of Foucault.

12  Circularity should be understood not only as being a property of dynamic systems, but also in reference to the circular model proposed by Marc Leman for the craft of musical instruments. In his book Embodied music cognition and mediation technology, Leman states that “[both] the actual process of instrument-making (ontogenesis) and the history of how an instrument evolves (phylogenesis) can be seen as the result of a repeated cycling of Play, Listen, Judge, Change.” (p.54)

13  A video teaser of the project gives a preview of the visual flow of the installation: https://vimeo.com/105187376 (visited 23 March 2017).

14 The wall of sound is a personal choice that comes from a particular affection for very low frequencies and immersive spaces, developed from a peculiar combination of incursions in the subterranean rivers in the Yucatan Peninsula on the one hand, and on the other, in the Metal-core and the Noise scenes, where I had the opportunity to experience the walls of sound of Zbigniew Karkowski, Blaster Kassandra and Helmut Schaefer, among others.

15  The visual flow of the installation was implemented with ISS/ISO Flock, a software developed at ICST, under the management and development of Daniel Bisig. «The Interactive Swarm Space (ISS) explores the application of swarm simulations for the creation of interactive and immersive spaces.» http://swarms.cc (visited 23 March 2017)

16  Frédéric Dufeu collaborated with me as a sound designer and a Max developper on the first version of Doppelgänger_Oblivion: Timeline, in Dieburg, on 2012.

17  Data bases and data structures are built with the object [coll], an object used in Max to create lists.

18  For real-time transformation of the additive synthesis flow, treatments and respective parameters are, for granulation: position jitter, grain length, distance, pitch, pitch jitter, amplitude, amplitude jitter ; for distortion 1: drive ; for distortion 2: drive, tone ; for delay: level, feedback, delay time ; for reverb: level, room size, time, absorption.

19  This, in turn implies a first level of compositional development in which the challenge, for the composer, is to conceive a program that induces the visitor into a dialogical relationship with sound, a subject that should be developed on a forthcoming article. The conception of this approach to composition implies that the composer should engage into different levels and modes of writing and programming.

20  For a detailed description see: MEZA, Mauricio, and Frédéric Dufeu, « Dopelgänger_Oblivion: Timeline. An interactive multimedia installation for collective in situ composition », Proceedings of the International Conference The Global Composition. Conference on Sound, Media, and the Environment, 459-461.Dieburg, Germany: 2012. https://www.academia.edu/attachments/43432162/download_file?st=MTQ5MDcxNjc4OCw4MC4xMS4yMDcuMjI4LDk4NDI5MQ%3D%3D&s=swp-toolbar (Viewed 28 March 2017)

21  I conceive my devices as if they would ideally run ad-infinitum. This idealisation should have it’s consequences concerning the intervals between a time of exposition and another. An online project is currently in development.

22 The idea of composition workshop should be understood as well as a convergence between a traditional, i.e analogical, workshop, and an electronic lutherie workshop, where the instrument, or meta-instrument, is actually composed, written and programmed. On the subject of composition as composition-of-the-instrument see: Bruno Bossis, «Ecriture instrumentale, écriture de l’instrument» (2007), where he addresses the subject from the perspective of an analysis of Jonathan Harvey’s Wagner Dream.

23 Doppelgänger_Oblivion: Timeline (2012), [L’emprise de Panurge]:::[Du feu de Toungouska] (2015), as well as the Semences d’écoute: Un théâtre des opérations (2016) where created with support of a grant by the National System of Creators of Art (Sistema Nacional de Creadores Artisticos, SNCA) and the National Foundation for Culture and Arts (Fondo Nacional para la Cultura y las Artes, FONCA), Mexico ; and with technical, logistical and human support by Césaré-CNCM, Reims, France.

24  An excerpt of the performance can be heard at https://soundcloud.com/mmcomposer/sets/emergent-instrument-project (viewed 23 March 2017)

25 The performance of the score was on the hands of Felix Del Tredici, at the international conference The Global Composition. Conference on Sound, Media, and the Environment, Dieburg, Germany, 2012

26 a2m is a composition tool created by the author of this paper in collaboration with Karim Barkati. http://karim.barkati.online.fr/Universite/Doctorat/Logiciels/ApplicationsMacOS/a2m/ (visited 23 March 2017)

27  A core notion in Gilbert Simondon’s ontology of individuation. «One could say that the principle of individuation is the common allagmatic operation of the matter and form through the actualization of potential energy. This energy is energy of a system; it can produce effects in all the points of the system in an equal way, it is available and is communicated. This operation rests on the singularity or the singularities of the concrete here and now; it envelops them and amplifies them.» Simondon, Gilbert, L’individu et sa genèse physico-biologique, p. 44.

«The Greek word allagma can mean change or vicissitude, but it can also mean that which can be given or taken in exchange, which more genuinely captures the idea of energy exchange in Simondon’s usage. Simondon will also define the allagmatic as “the theory of operations” (IGB, 263), complementary to the theory of structures that the sciences elaborate. On the same page, Simondon will define an operation as “a conversion of a structure into another structure.”» Weissman, Joseph, Fractal Ontology blog https://fractalontology.wordpress.com/2007/11/28/a-short-list-of-gilbert-simondons-vocabulary/#more-342 (visited 23 March 2017)