Download the pdf version HERE
I started by doing research on digital instruments design, where a good resource for research is New Digital Musical Instruments[1], which presents an extensive analysis of a wide range of projects. Although the frameworks presented are particularly focused on instruments rather than on experiences, most of their principles can be applied to my project as analysis tools. The authors state that there are six steps for designing DMI’s, outlined below:
- Decide on the gestures used as controls
- Decide the gesture capture strategies
- Define sound synthesis algorithms
- Map the sensor outputs to the synthesis and music-control inputs
- Decide on feedback modalities
GESTURES
Gestures can be classified into two types according of their physical interaction with the controls. The Semiotic (or Naked Gestures) and the Ergotic (or Manipulative Gestures)[2]. On the former, there is no direct physical contact with the device that crates the sound. An example for this would be David Rokeby’s Very Nervous System, where sound is controlled through a videocamera reading body movements. In contrast, an ergotic interface could be any traditional instrument, or the Reactable, where there is some sort of physical contact with the device.
Most of the prototypes I’ve constructed have explored semiotic gestures, so I thought it would be valuable to try ergotic gestures at this stage. The Pyramid and Pendulum prototypes explore around this idea, where there is a direct interaction with the controllers, so the interaction is mainly ergotic. However, using computer vision techniques allow semiotic gestures to be involved as well, creating an interesting blend of two territories.
CAPTURE STRATEGIES
The Play Station Eye camera has been the choice for most of my prototypes because of its fairly good optics compared to a regular webcam. As a capture technique, color tracking has been successful so far, but I’m aware that it does not work well in low-light conditions. Using a camera that is capable of capturing infrared light could solve this issue, allowing any kind of light conditions. In this sense, the Microsoft’s Kinect camera features an infrared sensor so it worths experimenting with it. The Kinect also features a depth sensor based on laser beams, providing a new spatial dimension that can be used as control. I will definitely test how this camera behaves in the following prototypes.
SOUND SYNTHESIS
Throughout the prototype’s iterative process, I’ve created a model that has given good results for audio synthesis. As explained in the Capture Strategies section, the gestures are captured using camera vision, more specifically, color tracking techniques. A program written in openFrameworks[3] analyses the data obtained from the tracking points and maps this information into parameters. The variations of these parameters are translated into messages and sent–using OSC protocol–to a second software called Osculator[4], that transforms this data into midi messages. A third software, capable of receiving midi messages from other softwares called Ableton Live[5], is used to produce sound. It was chosen for its capacity to easily produce controllable sound loops with a good sound quality.
MAPPING
So far, trying different mapping alternatives has been one of the main differences between prototypes. Probably the way the gestures are mapped into sound is the most influential variable, and it profoundly impacts on the kind of interaction: it sets the rules, affordances and constraints for the experience. Although a good amount of alternatives have been explored, I cannot make a definitive commitment to a certain mapping at this point.
However, one of the most interesting experiences took place with prototype # 5, the Pendulum, that incorporates–for the first time–the use of physical objects operated by participants. I plan to further explore the use of objects that inherently imply physical challenges of operation.
FEEDBACK
To balance visual and sonic feedback is challenging. Visual feedback is very useful for understanding how the system works, but an excessive presence can diminish the level of attention between players. When the visual feedback is strong or extremely seductive, people tend to give most of their attention to it, diminishing awareness about the musical and social experience[6]. Taking this into account, my project will use primarily sound feedback, making use of the visual aspects as a complement for the sound experience.
Having a system that is highly responsive to the participants’ gestures will be extremely important. As novice players are the main target, the musical range will be highly constrained, but this can be balanced by providing different control modalities, like controlling low-pass filters. A good balance between constraints and controls will allow to have expressive and very reactive feedback to users’ gestures.
THE MISSING VARIABLE
As the model I’ve been using for this section aims specifically for DMI’s, when applied to experiential design, it lacks of a last step: choosing a proper context. I feel that as my goal is not to create an instrument, but more an interactive sound space, the context where the experience takes place is as relevant as the interface or the sound itself. When I performed tests with Trinidad, I realized that this was an extremely important component to provide the appropriate conditions for the experience. I can see now that the interface is not to be designed alone, but as a tandem with the space that contains it.
Some questions arise here: How large needs the space to be? What are the lighting conditions? What are the sound conditions? Will the space take an active or a passive role in the interaction (i.e. the light conditions can be controlled by the participants, as a complementary reaction to the inputs).
In conclusion, some of the decisions I’ve made during this stage are: Will continue exploring the use of physical controllers, as the interaction seems to be more appealing when they are present. Mapping, while reducing the range of musical possibilities (melodies), must augment the sense of control and expressiveness. The context must be adequate for the scale (number of participants), and encourage intimate interactions.
___________________
[1] Miranda, Eduardo R. and Wanderley, Marcelo M. New Digital Musical Instruments: Control and Interaction Beyond the Keyboard. Wisconsin: A-R Editions, 2006.
[2] Miranda, Eduardo R. and Wanderley, Marcelo M. New Digital Musical Instruments: Control and Interaction Beyond the Keyboard. Wisconsin: A-R Editions, 2006.
[3] http://www.openframeworks.cc/
[4] http://www.osculator.net/
[5] http://www.ableton.com/live-8
[6] Blaine, T. and Perkis, T. The Jam-O-Drum Interactive Music System: A Study in Interaction Design. In Proceedings of Symposium on Designing Interactive Systems. 2000, 165-173.
On the previous section I identified three types of roles for interfaces: intercommunication, mediation and facilitation. Intercommunication role will not be explored in my project, that said condition is not adequate for the general aim of this thesis, where communication between participants is priority. Consequently, the ground between mediation and facilitation is the direction I will pursue for designing the system.
I also have discussed how the experience should be in terms of interaction. However,a remaining question is: What is this for? There is a wide range of expectations towards the musical experience that this project could address. Ranging from entertainment to musical performance, the wide spectrum of expectations deeply influence on the characteristics of the interface. The final product could then range from a toy–in the case of entertainment–to a musical instrument in the case of music performance. Although at this point I cannot say exactly what my project will be, I am certain about what I don’t want: is not going to be a toy, and is not going to be an instrument.
Since this project is targeting group experiences, the internal social relationships between participants need special attention. The question here is: Who will participants collaborate with? It is completely different if they collaborate with their couples, with their bosses or with a complete strangers. Different levels of intimacy between participants require different contexts, features and roles for the interface. Trying to understand where my project should fall in this wide social spectrum, I created a bi-dimensional graph plotting level of intimacy and context. These two conditions influence on each other, so it is important to find the most adequate combination of both. For instance, an interface meant to be used in the living-room would be completely different from one that is set up in a public plaza. The decision to work in the public or in the private sphere is relevant (and difficult!) so I’m not completely decided on this yet, however making this graph helps me to narrow the scope of possibilities by choosing an area.
Making research on social gatherings in diverse contexts, one of the social activities that caught my attention are rituals of Native Americans around drum circles. In this rituals, many of the things I’m interested in are present: collaboration around a sound context, communication between participants, a sense of belonging to a community with common goals. The interaction is interesting too, a very simple gesture–hitting the drum in a constant pace–creates something really powerful and beautiful: humans singing to the spiritual world. Here, no musical expertise is really needed and the most important thing is the social experience.
Another example are community music groups. There are some institutions that create activities around music for members of a given community, where people with different levels of musical expertise can participate in collective experiences. One of the most noticeable institutions is the Developmental Community Music[1], that organize experiences for different communities such as neighborhoods, office spaces, schools and any kind of community interested in participating. The goal of DCM is to facilitate musical group experiences with the aim of developing self-improvement, communicational skills, teamwork skills and spiritual fulfillment. The power of sound.
Getting into spaces, I started looking to places that create intimacy and behavioral change, trying to understand what are the conditions to allow this. The karaoke bars and booths–so popular in Japan–are perfect examples of spaces that meet those conditions. Karaoke spaces are usually used by groups of friends looking for an entertaining experience. Within this space, people behave in a completely different way as they would do outside; entering a karaoke booth implies some behavioral rules responding to special rules that exist only inside the room. As described By Salen and Zimmerman[2], a Magic Circle has its own set of rules, and these rules create a parallel space of reality:
“…Within the magic circle, special meanings accrue and cluster around objects and behaviors. In effect, a new reality is created, defined by the rules of the game and inhabited by its players.”
So, what conditions make spaces become a place for intimacy and behavioral change? It seems that the basic variables are: light conditions, area of space, sound conditions, space distribution and services. In a karaoke space for instance we have dim lights, a closed space (proportional to the number of participants), a good sound quality, a right distribution of furniture that encourages the activity, and food and drinks at hand. The combination of all these provide the necessary environment that karaoke requires.
In a similar fashion, Photo-Booths are also spaces with their own behavioral rules. It is interesting that although all what these booths do is nothing more than taking a set of pictures, using one is usually a fun experience and it produces a profound behavioral change on the participants. There is something very unique about entering a very tight box, making people share their personal space with others. Again, the photo booth conditions: dim lights, very tight space, small seats, and a curtain that separates the booth-world from the ordinary. In this case, just like the karaoke example, the interface sets the rules and drives the behavior; the time between each picture, the amount of pictures, the view area of the camera, are things that encourage people behave in a certain way for a certain period of time.
I want to point out something I saw one night at a local bar in Brooklyn: a group of ten to twelve people were playing traditional American music. They were not officialy a band, they were people getting together to make music. Following the tradition of the musical style, violins, mandolins, double basses, guitars, violas, and the essential banjo were part of the ensemble. People were displaced in a circular shape, distributing along various layers of distance from the center. Closer to the center, the more experienced musicians played their instruments, conducting the performance. Moving away from the center, a second layer of intermediate musicians followed the ones in the center, assuming a supporting role to the ensemble.
On a third layer, behind the intermediate layer, people that either were learning how to play the instrument or were getting to know the style also assume supporting roles, sometimes just observing and listening to the music. I had the chance to talk to them, and they explained that they were not a band. They were people meeting every two weeks to play together, “The group is different every time we meet depending on who shows up, we always have new people that we never met before” said one of the participants. The most experienced musicians have their own bands, they do this because it’s fun and for meeting and sharing with people with similar interests. “Anyone with an instrument and can participate and learn.”
The beauty of these meetings is that novices and experts come together, and feel comfortable in the same experience. Even though is clear that there are different levels of participation according to the different skill levels, socially everyone is equal, and everyone’s invited. I witnessed that myself: when they ended playing a couple of songs, one of the guys in the center introduced himself with the one sitting on his side, after talking a while, they exchanged their guitars, this seemed to me a welcoming ritual for a stranger.
The findings from this section were very enlightening, now I have a better understanding about the conditions of spaces that encourage collaborative interactions. The idea of a booth remains in my mind, and definitely some of their characteristics will be incorporated into my project.
_____________________
[1] http://playsinglaugh.com
[2] Salen, Katie and Zimmerman, Eric. Rules of play: Game Design Fundamentals. London: The MIT Press, 2004.
Based on the methodology described in the previous section, I’m currently focused on defining the characteristics of the experience. Putting it in simple words: What do I want people to experience with my project? What kind of interactions will the system require? How can interactional requirements shape the social experience?
Doing research on the HCI aspects of these experiences, will allow me to better understand in what ways the interaction between participants and the system can influence on the overall experience, setting different moods. Understanding this better will help me to later take decisions about the target users, interface characteristics and context requirements. By doing research on different projects that relate–directly or indirectly–to my own, I can get a better idea of all the variables that need attention, what are the common points, and goals. In short, is evaluating others’ projects so I can see the strengths and pitfalls that are common in this field and apply those findings into my own project.
Since the 1950‘s there has been an increasing interest on exploring collaborative sound experiences using non-traditional instruments, examples as John Cage’s Symphony of 12 radios [1] were trying to push the traditional boundaries of what was known as collective performances. Since the 70’s, when computational systems became more accessible, artists and scientists have been able to let go their imagination seeking for new ways of interacting with new musical devices and systems. In the last ten years, there has been strong interest among the academic and design communities on this subject. The existence of NIME, the annual conference on New Interfaces for Musical Expression is a proof of that.
Today, the current state of technology and the influence of DIY culture, are allowing more people to explore new approaches to collaborative musical expression, and new projects emerge every time more rapidly. So, my path is not unique, I’m obviously not the first one to investigate this issues nor the only one with the same objectives and passion about this. The good thing is that there’s a community and I’m not alone. I here quote Scott Pobiner, my thesis Professor: “stand in the shoulder of giants”–he said. The challenge is then to find how this thesis contributes to the field. How will it be different from other projects? How will it be better? These are some of the questions that I need to address in order to make this thesis unique.
A study presented for NIME 20102 outlines a broad classification of experiences of ISM (Interactional Sound and Music) by breaking them out into three dimensions: By situation: co-located or remote collaboration, by focus: task-oriented or creative engagement (improvisation), and by immediacy: real-time sound generation or sound-editing. Although the categorization presented here is very useful, I see it with critical eyes because it posits the focus as a binary framework. I believe that having an experience that demands the completion of tasks does not exclude creative engagement, and should be space for both approaches to coexist. However, this statement requires further investigation, developing prototypes will be a way of submitting this classification to test, to see how this relation works. I’m really interested in the following question: can a task-oriented interface allow space for improvisation? My gut feeling says it should, but I won’t be sure until I research more on this point.
Based on my research on different projects, I think that the role that the interface assumes can be classified in three categories: intercommunication, mediation and facilitation. The role of intercommunication implies that most of the communication between participants is done through the interface. For example, in the case of console games such as the Microsoft’s Dance Central designed for the Kinect, the interaction encourages participants to focus their attention to the screen, and all the messages sent between each other are mediated by a digital representation the players. Participants and their interactions become part of the avatar world, making the physical world less important than the virtual.
The second type of role is mediation, where the interface influences and shapes–up to a certain extent– the communication between participants. An example for this would be the Jam-O-Drum [3], where the interface sets the rules for communication but participants can still generate direct interactions between each other. The third type is facilitation, where the interface is in the periphery of attention, and plays the role of enhancing or catalyzing direct communication. An example for this case could be Lucas Werthein’s installation Samba Surdo [4], where sound is the only feedback and the internal mechanics of the interface are hidden from the participants’ attention.
To conclude, some of the main questions that arise from this section are: Can the experience be rich in communicational terms when the interface is mediating the interaction? Can improvisation (creative engagement) be fostered with a task-oriented experience? How can participants’ behaviors can be influenced or modified by the interface?
_____________________
[1] http://www.guardian.co.uk/music/2009/aug/06/john-cage-symphony-for-radios
[2] Bryan-Kinns, N., Fencott, R., Metatla, O., Nabavian, S., & Sheridan, J. (2010). Interactional Sound and Music: Listening to CSCW, Sonification, and Sound Art (Demo/Poster). In Proceedings of NIME 2010, Sydney, Australia.
[3] Blaine, Tina, and Tim Perkis. 2000. The Jam-O-Drum interactive music system: a study in interaction design. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, 165–173. DIS ’00. New York, NY, USA: ACM.
[4] http://www.lucaswerthein.com/?p=227
I read some texts from Tina Blaine at the beginning of this process, most of them analyze different projects from various authors, but she also has designed a prolific amount of musical interfaces. One of the most relevant for my investigation is the Jam-O-Drum, an interactive tabletop interface that can be considered as a predecessor of the Reactable. Blaine along with her collaborator Tim Perkins wrote a paper for DIS 2000 [1], where they explain in detail the process of designing the system: they divided the study into two initial phases. On the first, they performed tests to define the interactional and graphical aspects separately. In Phase Two, the results from Phase one were incorporated into an integrated model, testing again the results.
My plan is to incorporate a similar iterative methodology, testing specific aspects of interaction through prototypes, and by the end of the semester integrate all the individual findings into a final, integrated proposal.
Prototyping will be an important aspect for my research: through user’s reactions to these prototypes, I will able to see what are the most suitable solutions. Prototypes will become then the spinal cord, the main road that will lead to answers and discoveries, and eliminate or confirm assumptions.
I’m framing my research into three main areas: The experience, the interface and the context. My intuition says that defining the experience first will lead naturally into taking decisions about the users, interface and space that supports the experience.
So inspired by Blaine’s model, the sequence should be divided into three steps:
- Define what kind of experience
- Define users
- Design an interface to facilitate the defined experience for the defined users.
Along the multiple examples I’ve studied doing research, I’ve seen a trending challenge when designing interactive sound systems for novices: balancing the level between expressivity and complexity. It is of the most importance, as this balance determines the entry-level. Let’s take an example from traditional instruments: a xylophone requires a simpler gesture than a violin, but so are the expressive possibilities of creating different timbres. Conversely, mastering a violin is harder than mastering a xylophone, but the expressive range of the instrument is wider. This is a trade-off I need to be address carefully, because my goal is to provide a very low entry-level, but to make the interface as expressive as possible.
Probably the entry level is one of the most important things I will have to put attention to. A different issue that also deals with this idea is to decide if this will be exclusively designed for novices. What I think it would be the perfect situation is a space where novices and experts can share the same experience, having similar amounts of joy. Maintaining the state of flow for both levels of expertise is a difficult task, and I feel that the answer is focusing more on the experience rather than in the musical aspects.
____________________
[1] Blaine, Tina, and Tim Perkis. 2000. The Jam-O-Drum interactive music system: a study in interaction design. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, 165–173. DIS ’00. New York, NY, USA: ACM.
In this section, I identify the different disciplines that will be relevant for my research. Not necessarily looking for answers within these disciplines at this stage, I’m focusing on visualizing the scope of my research. Although this list will definitely be growing over time, this is where I’m starting.
On the humanistic side, psychology will help me understand the social dynamics between participants, thus is very important for this research. There is a vast body of research around HCI (Human Computer interaction), a subset of psychology. This discipline is relevant because it deals with how humans interact with artificial systems, and what are the psychological aspects within that interaction. CSCW (Computer Supported Cooperative Work), an even more specific area of HCI, studies how people interact collaboratively through computer-mediated systems. CSCW addresses the concerns and rules for designing a system to generate the desired behaviors.
On the design realm, looking into game design, software and musical instruments design will help understand the variables for designing a system that deals, at the same time, with all of their concerns. On the technical side, spatial design and sound design will play their role in determining the kind of space and sound conditions to support the experience. As I could realize when testing Trinidad, the context is very important as it influences on people’s predisposition to behavioral changes. Just imagine a dance club with bright, fluorescent white lights, it simply wouldn’t work.
At this stage of research, I am focusing more on the humanistic side. It makes sense to me that a good first step is understanding how people operate in experiences that require collaboration. What are the conditions needed to generate engagement with an activity? What influences the communication between participants? How does flow works?
In The Psychology of Happiness [1], Michael Argyle refers to a chart from the American Psychological Association, that plots the classification of moods. It maps all the adjectives associated to moods into a positive and negative scale, crossing the dimensions of pleasant-unpleasant and arousal-sleep. This maps broadens the scope of what I was considering a “fun” experience, breaking it down into more specific states. There are different kinds of pleasurable experiences, we sometimes experience as much pleasure when staring at a sunset in a calm beach as we do when playing Black Jack at the casino. Same feeling, but shaped in an entirely different way. This brings me to the question: What kind of pleasurable experience I’m looking for? Is it going to be something that makes you feel relaxed or something that makes you feel euphoric? Both sides are interesting, but it is important to notice that aiming to one side or the other changes everything. I don’t have the answer at this point yet, but having this in mind is helpful to define an initial area to explore through prototyping.
I also did further research about the state of flow. According to Csikszentmihalyi [2], this state can be visualized as a channel that arises when skills and challenges are properly balanced.
“ In our studies, we found that every flow activity, whether it involved competition, chance, or any other dimension of experience, had this in common: It provided a sense of discovery, a creative feeling of transporting the person into a new reality. It pushed the person to higher levels of performance, and led to previously undreamed-of states of consciousness. In short, it transformed the self by making it more complex. In this growth of the self lies the key to flow activities.” (Csikszentmihalyi 1990, 72-77)
Taking these ideas from different studies, mostly from Csikszentmihalyi, it can be said that there are four conditions for systems to facilitate flow:
- Contain rules that require the learning of skills
- Set up goals
- Provide feedback
- Make control possible
___________________
[1] Argyle, Michael. The Psychology of Happiness. New York: Methuen&Co.,1987.
[2] Csikszentmihalyi, Mihaly. Flow: The Psychology of Optimal Experience. New York: Harper & Row, 1990.
SOCIAL ACTIVITIES
A great place where I can explore the social dynamics involved in these experiences are the leisure activities we perform in our regular life, particularly the ones performed in collaboration, like dancing, practicing collective sports or playing board games. All of them have common dynamics that can be applied to my own project. These activities and their dynamics will be further investigated throughout my research process.
IN THE ART REALM
I find the work of David Rokeby very interesting and I feel it is relevant for my research. Rokeby is an artist that has explored the relationships between body, sound and physical space. Since the eighties he has developed a series of installations that make use of technology to convey these ideas. One of his most interesting pieces is Very Nervous System [1], an installation that uses computer vision to interpret body movements into sound, creating very complex and expressive compositions. My interest in his work relies on how he uses the body as an expressive controller. Additionally, the technical aspects of this project are relevant too. Using camera vision techniques is something I feel valuable to explore.
In contrast to my own exploration, Very Nervous System is meant only for one participant, and mainly for experts–although experts in this case would be dancers and not necessarily musicians, which is interesting too–. I also would like to differ from the experience mood he presents, I’m not interested in introspective interactions, because I feel that is not the most adequate mood that best suits a collaborative experience.
SIMILAR PROJECTS
An inspiration from the realm of collaborative musical instruments is The Reactable [2], a tabletop synthesizer using physical objects as controllers. Although I’m sure is one of the most cited examples in these kind of research, a lot of that research con be incorporated to my own. In the Reactable project, the collaborative aspects are really well executed. The idea of a round table for instance, implies the idea of democracy, where all the participants have the same hierarchy.
Although the Reactable is pitched both as an instrument for novices and expert performers [3], it seemed to me–I had the chance to play it–that is more an instrument leaning towards expert performance. In the practice, the Reactable is not that easy to play o understand, and as any other traditional instrument, it requires considerable practice to be mastered. Also, the visual feedback is very seductive, making the participants be completely absorbed by the visual, not worrying too much about the human-to-human interaction.
THEO WATSON
I find the work of Theo Watson very inspiring. In particular the Vinyl Workout [4] and Audio Space[5] installations. Both make a use of space and sound in interesting ways. What I find remarkable about Vinyl Workout is its simplicity. It consists on a vinyl record image projected on the floor. Users walk inside the projection in circles, making the vinyl spin and play songs in different speeds depending on how fast or slow participants walk. The beauty of the system is that although it requires a very simple gesture, it generates interesting and expressive interactions. Unlike Rokeby’s Very Nervous System, Audiospace uses the three dimensions of space, where sounds are mapped to specific locations in a room. As users walk around the room with a pair of headphones, they can hear the different sounds and perceive their virtual position, as if they were floating in space. Again, the mood of the experience is introspective and very personal (users wear headphones), which is not exactly what I’m looking for.
___________________
[1] “Very Nervous System,” David Rokeby, accessed September 20, 2011, http://homepage.mac.com/davidrokeby/vns.html
[2] “The Reactable”, Reactable Systems, accessed September 21, 2011, http://www.reactable.com
[3] Kaltenbrunner, M., Jordà, S., Geiger, G., and Alonso, M. The reacTable*: A Collaborative Musical Instrument. In Proceedings of WETICE. 2006, 406-411.
[4] “Vinyl Workout,” Theo Watson, accessed September 22, 2011, http://www.theowatson.com/site_docs/work.php?id=39
[5] “Audio Space,” Theo Watson, accessed September 22, 2011, http://www.theowatson.com/site_docs/work.php?id=15
In this phase of the thesis, I decided to start with the main questions and goals. Questions as: Why? What is the problem? Who cares? Are the leading concerns at this stage. Taking the previous experience of Trinidad as a starting point, and revising the outcomes, in this section I try to define what are the main frameworks where my thesis and project will reside, with the aim to visualize a blueprint of the different areas I will be researching during the thesis year.
Daniel J. Levitin in his enlightening book This is Your Brain on Music [1] exposes the fact that in our times–specially in Western societies–music expression is most of the times exclusively reserved for experts:
“Only relatively recently in our own culture, five hundred or so ago, did a distinction arise that cut society in two, forming separate classes of music performers and music listeners. Throughout most of the world and for most of human history, music making was as natural an activity as breathing and walking, and everyone participated. Concert halls, dedicated to the performance of music, arose only in the last several centuries.” (p.6)
I don’t consider myself a virtuoso instrumentalist, or even a musician. Musical notation is as foreign as Russian language is to me, and I don’t really understand musical scales. Yet, I’ve been making music for more than fifteen years, and I’ve managed to play in some bands. My sense is that most of musicians don’t really care about the technical aspects of music, they make music because they like it, because it is a rewarding and pleasurable activity and playing with someone else just amplifies those feelings. So I wonder: why the distinction between performers and listeners is so prevalent, when making music is one of the things that fundamentally defines us as humans?
Although people without musical training experience flow in several other contexts, they usually don’t experience it around music creation. One of the main reasons for this is that the learning process of mastering an instrument requires practice, dedication, patience and time. A violin for instance, can take several years to be played with mastery. “I don’t know how to play an instrument, therefore, I can’t participate in music exploration” is the thought sequence that most people follow.
Around this idea, is where my thesis resides and builds the main concerns: How can novices get involved in collaborative musical expression? How can participants experience the state of flow around music creation? What role can play a Design and Technology thesis to address this issue? How can technology facilitate these experiences?
My thesis then will investigate the social interactions in collaborative sound environments, having as the main design question: How can technology facilitate collaborative sound experiences? The aim of the project that illustrates this thesis is to provide a structure where participants are encouraged to achieve a loose state of mind and openness towards sound exploration. This state of mind sets the basis for the development of collaborative behaviors, enhancing the social cohesion between participants that ultimately leading to experience group flow. Setting up small sets of rules for the interaction, the structure should allow emergent play and musical expression, but these should be considered only as means to achieve the desired social experience.
Consequently, the main conceptual framework will then be around these ideas. In the core of said framework is collaboration, where all the components such as the system that supports the experience will aim. Sound, in a second level, acts as one of the means to achieve the social experience. Play, on the other side, acts as a catalyst, and technology, acts as a facilitator for the interaction between people and the whole system.
Finally I outline here some of the key-points or goals for my project:
- Facilitate playful interactions
- Provide a platform for exploration
- Provide a small set of rules
- Allow low level-entry to the sound experience
- Create a fertile ground for the sequence of exploration-discovery-learning-collaboration
_______________________
[1] Levitin, Daniel. This is your brain on music : the science of human obsession. New York N.Y.: Dutton, 2006
Inspired by a trio configuration–which is very common in Jazz and Rock–Trinidad is operated by three participants. Following the trio model of guitar-bass-drums and their individual roles within the music ensemble, in Trinidad one player performs the melody, a second one performs the base, and the third player performs the rhythm.
My initial idea for Trinidad was to create an audiovisual system that allowed novices to play as a band. My assumption was that participants, when confronted to an interface that makes music performance accessible for everyone, would start jamming, trying to create different musical phrases and moods, just as experts would do with traditional instruments. Although this worked out up to a certain extent, after the initial prototype tests, I realized that my assumption was not totally accurate. People didn’t care about creating “great music” as much as they cared about having a good time together. It opened my eyes to the idea that although the musical outcome was important to provide grounding and facilitation, it was not as important as the social experience. This was a groundbreaking discovery that had a profound impact on the direction of the project in several ways: I was no longer interested in making an instrument but an experience, where despite the expertise and the musical expectations, everyone could be able to participate, have fun doing it, and ultimately experiencing flow. After all, isn’t that the point of making music?
Taking this idea into account, for the final user testing I chose a proper context, paying attention to the sound and lights conditions as well to the graphic style and general look of the interface, trying to provide fertile conditions for the social behaviors I was looking for. I tested this with three groups, all of them classmates, some of them knew a bit about my project and some of them didn’t, but none of them were musicians. The overall experience was successful: participants had fun, they communicated between each other in verbal and non-verbal ways, and they were completely focused on the experience. It was extremely interesting to see what happened in that test and I was delighted to be able to experience it from an observer position.
During the hour and a half that the experience lasted, I could observe a common pattern in the way the experience evolved in each of the three groups. First, they explored their own individual space, figuring out how to operate the interface: understanding how the gestural inputs were translated to sonic and visual output, discovering how certain gestures produced different outcomes. Once they learned and felt comfortable about their individual role in the whole system, they started communicating with the rest of the participants, most of the times being curious about each other’s sounds or about what the others were doing. Communication allowed sharing knowledge, teaching each other about the features of the individual sounds and controls. The last step was trying to make something together, coordinating and synchronizing to create a common outcome.
After all these steps, the most interesting phenomenon happened, they started playing with the interface in different ways, for instance, occupying the individual sound space of a partner without permission, jumping around in circles or trying to play–all together–the same sound at the same time. They were totally breaking the implicit rules of the interface and behaving as children.
I find this last thing very interesting: given a very basic and simple set of rules, the system allows the users to explore the interface, but most importantly, to explore the territories beyond the ruleset. As Salen and Zimmerman explain in The Rules of Play[5] , emergent play is the play that arises from breaking the original ruleset of a game, creating then a new kind of game, becoming an endless loop that makes play evolve.
Most of the ideas from Trinidad are being carried to my current thesis, my plan is to keep exploring this phenomena around sound experiences: how people communicate, how they build up strategies to coordinate and synchronize, how they break the rules of the given system and create play, and how a joyful collective experience leads to the state of flow.
More details on Trinidad here : http://franciscozamorano.cl/?p=1067
__________________________
[1] Salen, Katie and Zimmermann, Eric. Rules of play: game design fundamentals. Cambridge Mass.: MIT Press, 2003.
My current exploration starts last year with my final project for Major Studio II. Pitched as a mini-thesis, it became an initial approach to the current concerns of my thesis. I started that project reading my Statement of Purpose that I wrote to apply to the MFADT program. One of the main goals explained there, was to develop projects integrating education, music and animation, responding to my different interests and professional background: I worked as a film director in a production company for several years developing motion-graphic-based projects, I had been teaching design at various universities, and as a hobby, I’ve been playing music for more than fifteen years. It made sense to me that these interests that have been part of my life needed to come together in a common project, that would at the same time, satisfy my own artistic curiosity and become something meaningful for others.
With that goal in mind, I developed a project called Trinidad, a collaborative sound and visual interface, that allows people with no previous musical training to experience musical expression as a group. The limited time for developing the project completely influenced on the way I approached it. Since the subject was novel to me, I operated mostly by intuition.
This was extremely positive, it allowed me to explore a virgin territory without prejudice, with complete openness and sometimes, positive ingenuity. But despite this intuitive approach, I did some interesting research, and I found out that there is a large body of research around the field of collaboration and cooperative performance in sound environments. One of the most revealing materials is an article from the Journal of New Music Research (2003) by Tina Blaine and Sidney Fels. Collaborative Musical Experiences for Novices [1] outlines the main aspects, concerns and outcomes for developing collaborative sound systems for non-musicians. It became some kind of bible for me, and up to the present day I feel it is one of the most valuable documents for my research. Equally relevant is the book Flow: The Psychology of optimal Performance by the psychologist Mihaly Csikszentmihalyi [2] where he coins the term Flow, a mental state of complete energized focus in a certain activity. In a TED talk, Csikszentmihalyi cites a music composer he interviewed regarding the state of flow around music:
You are in an ecstatic state to such a point that you feel as though you almost don’t exist. I have experienced this time and again. My hand seems devoid of myself, and I have nothing to do with what is happening. I just sit there watching it in a state of awe and wonderment. And [the music] just flows out of itself.[3]
I have experienced this myself when I used to perform music with other people. Most of musicians know about the unique state that is reached in the music sessions: you are not focused on yourself, you are focused on the collective action, on the others, on the music. Your self-consciousness is reduced and the perception of the synergy generated as a group is enhanced. There’s a pleasurable rewarding feeling about the idea of creating something together, becoming then some kind of group ritual that–using Victor Turner’s terms–enhances the sense of communitas.[4]
__________________
[1] Blaine, Tina, and Fels, Sidney. 2003. Collaborative musical experiences for novices. Journal of New Music Research 32 (4) (12): 411-28.
[2] Csikszentmihalyi, Mihaly. Flow: The Psychology of Optimal Experience. New York: Harper & Row, 1990.
[3] Mihaly Csikszentmihalyi on flow | Video on TED.com”, n.d., http://www.ted.com/talks/mihaly_csikszentmihalyi_on_flow.html.
[4] Turner, Victor. The ritual process: structure and anti-structure. New York: Aldine de Gruyter, 1995.