Magda: Hello, everyone, and welcome to our first meeting of this year's Transmedia Arts Seminar, co-sponsored by Mahindra Humanities Center at Harvard and MetaLab at Harvard. And today, first, I want to introduce my colleague, Ramona Mosse, will be co-chairing the seminar with me this year. We are thrilled to have Ramona on board and I also want to introduce introduce Jingyi Zhang, who is a graduate student at Harvard Musicology and Jingyi will be leading the conversation today. We are excited. We are going to be talking about contemporary experimental opera and opera and technology and A.I. and digital experiments with operatic music.
So thank you, everyone, for joining in today. And after the conversation, we will have a Q&A and everybody will be able to ask our participants questions, either through chat or through Q&A feature. This seminar is recorded, just to remind everyone, and it will be posted and available on MetaLab and Mahindra website shortly.
All right. Thank you so much, everyone. Thank you, all of the participants. Jingyi, do you want to take it from here? Jingyi: Sure. So hi everyone. I'm Jingyi and really happy for everyone to join us today for this special event. So I'm the moderator for this really awesome panel today and we are truly honored to have such a wonderful group of scholars, practitioners joining us for our panel on the future of opera.
So I'll first begin by introducing the panelists in alphabetical order, starting with Jelena Novak. So I'm personally very fond of Jelena's work as a contemporary opera scholar. She's written a lot of books on this. So her fields of interest are modern and contemporary music, recent opera, singing, new media, capitalist realism voice studies and identities in music. Exploring those fields, she works as a researcher, lecturer, writer, dramaturg, music critic, editor and curator focused on bringing together critical theory and contemporary art.
Her most recent books are Operaphilia from 2018 and Einstein on the Beach, co-edited with John Richardson, published in 2019. Ellen Pearlman is a New York based new media artist, curator, critic and educator. She's also the founder and director of ThoughtWorks Arts, a global technology research lab.
She created Noor, a brain wave opera and AIBO, I hope I pronounced it correctly, which stands for an emotionally intelligent artificial intelligence brainwave opera. Currently, she's working on "Language is Leaving Me: An Opera of the Skin," which explores A.I. computer vision and the impacts of trauma in cultures of diaspora through epigenetics. Next, we have Kamala Sankaram, who is praised as one of the most exciting opera composers in the country. She moves freely between the worlds of experimental music and contemporary opera. Recent submissions, works for the Glimmer Glass Festival, Washington National Opera, and Creative Time, among others. Kamala is known for her operas based in Indian classical music and for work that pushes the boundaries of the operatic form.
And last but not least, we have Brian Staufenbiel, who directed and spearheaded the conceptual designs of the San Francisco company Opera Parallèle Productions Since it was founded in 2010. Specialized in multimedia, immersive and interdisciplinary productions, he actively works across a wide range of artistic disciplines, collaborating in film and with media designers, choreographers and dancers, circus effect artists and designers for theaters. So once again, we are really fortunate to gather this amazing group of multidisciplinary artists and scholars here with us today. So, a small note on our roundtable session today.
So as moderator, I will be posing questions to the whole group and then anyone from -- any panelist can jump in at any time to respond to them or to each other. There are also speaker-specific questions later on. And lastly, we will have time for a Q&A at the end. So let's begin. So it's been over two years and we are still stuck in this global pandemic. So perhaps I think it's a good idea to begin the session with a question on opera making today. So my first question for this panel is, the COVID 19 pandemic offered this opportunity for the performing arts to spearhead innovative transmedia storytelling formats that move fluidly across media boundaries, artistic genres and geographical borders. So how did it affect your own work and how did it impact other artists whom you know? So anyone can start the ball rolling. Kamala: Maybe I'll jump in.
I think that I think that I did one of the first of these experiments with sort of telematic opera using Zoom, which, for better or for worse, is a thing now, because I and many of my colleagues, our work was canceled for the foreseeable future, and I was concerned that it was actually going to last longer than six weeks, as was being said at the time. So I brought together a few different singer friends and asked if they would be willing to jump in and try to do something over Zoom. And what that led to was sort of finding a way to create a piece that had no -- that was not working from a shared sense of pulse, but was rather more aleatoric in nature. So there's an open sort of chance element to it. And it really gave the singers the kind of agency that they don't always get in opera. And I can show a short clip of that if you'd like. Jingyi: Sure.
Kamala: So this is -- this piece was called "All Decisions Will Be Made by Consensus." And as you'll see, it's about a Zoom meeting and it takes place in Zoom. >> [singing] I think we -- >> [singing] Oh, sorry. >> [singing] You go. >> [singing] I think we --
>> [singing] Sorry, I was muted. Sorry. >> [singing] You go. >> [singing] Sorry, I was muted. I'm taking minutes. Just so everybody knows. >> [singing] The one thing we need to decide today is a date for the action. >> [singing] Now is our moment. Right now. >> [singing] There was a question about the weather. There was a question about the seasons. >> [singing] I think we -- sorry. You go. Not everyone has the same bandwidth. This is not the way to go.
I was levitating the Pentagon before you were born. Kamala: So. What's interesting about working on this piece was that it kind of returned this agency to the singers in a way that we have moved away from in the field of opera in general. And so what I would like to see going forward is that more opera companies now are experimenting with digital platforms and with emerging technologies, but also putting the singer in the center in a way that we haven't always.
And I think that that's something else that's really important that's come out of the performance practices that have emerged in this time period is just that the singer, the performer, not only as an interpreter, but also as a co-collaborator, if that makes sense. Jingyi: That's awesome. It's amazing. I also --does anyone else want to chime in? Ellen: Yeah. I wanted to say something. I didn't have a clip prepared of this, but I made something called -- it wasn't -- didn't have singing because it wasn't opera, but it's very relevant to performative called dancedemic in a pandemic, in which one dancer alone was in a studio and turned on a camera and a biometric device was strapped to their body. And as they danced, they had collaborators creating the set design live time all around the world, and they had musicians collaborating live time with the soundtrack as their biometrics changed the visuals and the sound as they were dancing. And this all happened on the network live time. So we had collaborators from San Francisco, Toronto, Florida, London, Estonia,
Ukraine, Brazil and India, all live time, manipulating the set and the sounds. And this was a group effort and it took place with one performer in a studio. So the sounds, it wasn't exactly opera, but it was performative practices.
So unlike Kamala, whose work I admire very much and is really fantastic, this was just using the network itself as the point of conductivity and the human biometrics to drive aspects of that. Instead of the voice as the driver, it was the human biometrics as driver. But the voice could be the driver. Kamala: But could I also -- I think what's wonderful about your work, Ellen,
and what you're talking about is how people from all over the world are coming together to collaborate. And that's -- I think that's another thing that changed in the pandemic is more openness to these sort of cross geographical collaborations that we, for some reason, were hesitant to try, even though Zoom has existed as a platform since before the pandemic. But I think that's beautiful. Jingyi: Yes.
I just want you also really beautiful that even though, you know, the pandemic shut down a lot of live performing, but it kind of created this new way of performing, like bringing together people in a very different way across time zones, across various kinds of cultural borders. Geographical borders. So that's something that is truly empowering for creative artists. Brian: Well, also, to Kamala's piece, which is really cool, is you've got this issue of latency, right? So and so you have to be savvy like she was.
You've got a kind of beautiful tonality, but then there's an aleatory quality of the singers being able to come in and bump to each other, but it doesn't ruin this kind of written out, you know, chordal rhythmic structure that you might be able to pull off in a live performance. So that's the thing a lot of people are having problems with. That's a great solve. At least what you showed us was great. I was going to say thatI -- it allowed me, because we had -- the company I work for, Opera Parallele, we were going to do Harvey Milk, you know, we were bringing it back for its anniversary and then everything shut down. So it allowed me to explore something I've been fascinated with and started earlier on, the sort of the synergy of graphic novel operas and opera, and sort of that -- they share a lot in common in many ways, especially in the use of language.
And I can share a little clip of what we did. We made a graphic novel opera film that we were able to share with people. You see that? Is that full screen? Sorry if it's not. No? Sorry. Jingyi: We can see that. Brian: You can? Jingyi: Yeah, we can see that. Brian: Great. I'm going to share that. Can you see that?
[singing] I here. It's Rob Hall. Made it. Everest summit. It's 2:30. I'm here! Brian: And so one of the things that was really fun is it allowed us, just really quickly, is that we were able to work with each person individually, put them -- do motion capture, illustrate them. And so we were able to do it safely in times of COVID. Like we put them in a big bubble and filmed their face, filmed them in a film studio, and then sort of with all the pieces, put together a film.
That was the only reason we could pull it off, and that's why we were able to do that. And it was a great adventure. Jingyi: Yeah. I had opportunity to watch this over the pandemic. And I thought it was really a very fascinating way of opera making.
And also it changes a lot of -- I don't know, like I'm also wondering for creative artists, when you are using this sort of new visual idiom in this new piece, does it impact the way you're composing in terms of the sound and as well as the whole conception of the piece? Maybe, Kamala, do you want to speak more to that? Yeah, because I think one of the things that had to happen was an embrace of amplification finally. There's always been this, this sort of a debate in opera as to whether to mic or not. And I think it's a tool, you know, and so what had to happen is because we were moving through this virtual medium, everyone had to be miked. And so now I think there's, I hope there's less fear of it.
Because one of the things that miking does is it allows you to have this more declamatory kind of singing style. Like if you're, if you're having to sing at the Met and you have to reach the back of a 3000 seat house, then you can't get the speech rhythms the same way that you can if it's close miking. And so that's been really interesting to see how it's impacted the way that that I for one, am thinking about writing for -- and I, you know, I've always thought about amplification as part of the composition rather than as something that is put in later on. But it's interesting to see how that's also being put into these these projects like Everest, right, which is intended to be a big piece, you know, without microphones and, and then, but still learning how to mic that in order to record it perfectly and all that.
And I think it's something that we really needed to do. So I'm glad that it happened. Mm hmm. Brian: To your point there, I can think of four different grand opera productions, couple at the Met, where they might. It's just like I don't know if that's a direct correlation to what you're saying, but pretty amazing. Kamala: Yeah. Yeah, I think I think we're just, I think there's an openness to it now that we don't have to hide the mics anymore because it's fine.
Ellen: Yeah. And I want to jump in on something Brian brought up that council said, which is the issue of latency when you talk about the mic, and what's happening, that has happened, is it mediated space of signal processing? And I run into that a lot when I work with biometrics because you have to do a lot of signal processing in order to translate it into another form. And signal processing, even if it's through a mic and certainly on the zoom has an inherent latency that you can't overcome because of the hardware and software and networks.
And so the performative aspects must incorporate that latency very strongly. Otherwise, the audience, I found, the audience doesn't buy it. So the actual construction and composition has to understand the latency, which in the type of things I do, runs anywhere from 2 seconds to 11 seconds because of the type of signal processing I do. And so that is the responsibility of the composer and the performers.
The performers have to retrain for latency aspects as you said, Kamala, with mics and signal processing. And that was really great with the cartoons. I would watch that in a minute. Yeah, really wonderful. Jingyi: So I'm curious, can each of you talk to me about what -- you've been exposed to all new technologies during this Covid time, so which media are you most drawn to and why? And what is the most challenging to work with? And then the follow up question, another follow-up question is which ones do you think are here to stay? So this is really speculative. So, yeah. Ellen: I can take that.
The thing I'm going to say is, number one, all of them are here to stay. Number two is I have gone in working -- you know, unlike these other distinguished panelists who are deeply trained in opera, I am deeply trained in tech. So it's a little different aspect.
And I've gone from working with brainwaves to working with brainwaves and artificial, and now -- into artificial intelligence, and now I will be working with artificial intelligence, speech, and computer vision. And all of these technologies are emerging, will have no way back, and were accelerated and are being accelerated through the pandemic. Brian mentioned motion capture, which is another technology which is, I would say, undervalued and just as important. And where motion capture is being privileged at this moment is, unfortunately, in the metaverse. You'll see a lot more of that in the metaverse. And I have conflicted feelings about performance in purely virtual spaces.
I think hybrid is much more effective than purely virtual. So that's the quick view from a more technological point of view, as opposed to a strictly operatic traditional development point of view. Kamala: And if I -- I would jump in and just say that, I think that that's the view from the opera point of view also in that, you know, like I think that -- I have a couple of projects that I've done in virtual reality and I think that what's always missing from them is the social aspect of going to the theater. You know, that's part of why we go is to see other people and to see how they're reacting and to feel like we've had this shared experience. So I think that's why a lot of these projects are now going towards something that is more hybrid, like augmented reality or mixed reality rather than pure virtual reality or even pure digital video.
You know, at the end of the day, it's nice to be able to be there with other people. Jelena: Yeah. Maybe here to mention the piece by Michel van der Aa. It's called "Eight." It was not made during the pandemic, but it was made just prior to it, in 2019, I think. But in a way, it announced what awaited us later in pandemic, because that was a piece that could be seen only by one person at the time.
And everyone has to reserve the slot of 15 minutes. But when I did that, actually, I was not aware that there is more into it. When I appeared in the music hall in Amsterdam where The performance took place, actually the whole building was completely emptied. No one was there except one lady at the entrance and she said, just follow the arrows and climb to the small hole at the top of the building.
At that moment I thought that it's a public holiday or something because it's empty, but all that was part of the direction of the piece, actually. So you climb the stairs, you arrive to the to the small hole. And actually there you meet the predecessor, so to say, the person who is waiting for you to enter. And then you enter the interactive virtual reality installation to be the only person at that moment in the universe to see that opera. It was me for the first time to be in opera in that way, to be in virtual reality. And the most strange experience that I realized only afterwards was that this artist made me a central character of the piece, which I never, ever expected. So that provoked just all kinds of questions,
like what it means to be in opera alone today and how you become part of the opera, actually, as a performer, in a way. So that was the interesting experience, even before the pandemic. Kamala: And I think that's something that we always have to take into consideration when dealing with emerging technologies, is why and what is the audience relationship to the technology. And like my piece, The Parksville Murders, which is the VR piece that I made in 2016, I think, the way that we dealt with that was the audience is the killer. Not to give it away, but it's a it's a horror piece. And so at the end, you as the viewer inside of it, you're the one who's actually killing people.
I think that's the tricky thing about VR, though, because you're in this, such an immersive, you're cast inside of it. Then we have to ask, why are you there? Whereas with -- I think with augmented reality, one of the things that's a little easier about it is that you're not in this, you have your own perspective rather than this perspective that's been being given by the virtual world. But I'm curious to hear what other people think about this. Brian: I definitely think that I'm excited to use technology and see what people are doing to
bring the audience more into the piece a little bit, what you were just both talking about, yet it can be dangerous and it can isolate people, too, right? If you put a headset on and you're by yourself at home and you're experiencing opera, that's, you know, the counter of having a roomful of people experiencing it and riffing off of each other and feeling that energy. So I've been very interested in seeing --and I'm both seeing it and experimenting in actual true immersive environments where the audience is inside of the piece. And we're actually going to take The Assets of Everest next season, and we're going to do a fully immersive 360 projected on the people with white ponchos, characters moving around. So you're like in base camp on Everest, you know, the audience. And that's where the technology is affordable. I mean, you guys remember when projectors were, to rent a one projector ten years ago was your whole budget. And now you can rent, for the same price
you can rent ten. So it's that kind of affordability is also helping us. Jingyi: Mm hmm. So all of you mentioned, you know, the importance of the audience experience and how it kind of places you in a very different kind of way to interact with a piece. So, so my next question for the panel is how does transmedia opera transform audience experience in different ways? Because in many of your bio works, the word immersive keeps coming up.
And a lot of, you know, many times certain operas only say is it's immersive production, and a site specific opera is immersive production, a digital opera is an immersive production. So wondering if you can share with us what immersiveness means means to you and why is there so much hype over the word immersive? And what are some elements that contribute to immersive experience in audiences? Jelena: I think first and foremost, immersive for me means that I'm inside of the piece because normally in more classical pieces of opera or just music, you just go there and you're kind of spectator from the outside, in a way. But when that position is designed in a way that you're even sometimes at the center of the piece, then you're immersed. But it does not have to be always with all these screens and all these experimenting with VR. For me, I think the most interesting things are when it happens only with sound.
And for example, with a production of Stockhausen's Licht in Amsterdam in 2019, I think also before the pandemic, Pierre Audi staged almost the whole cycle, but with truly magnificent surrounding sound in the Gas Fabrique in Amsterdam. And I think that was one of the most immersive experiences that I felt in opera, although the role of the screens and projections, visual projects was not big at all in that production. But the role of the sound and the sound projections was really something interesting in that piece. Brian: I'll just jump in that I've had many conversations with people about what is just like what you asked, right? Because some people really feel immersive, the audience has to have some sort of agency for it to truly be immersive in the theater sense, where you're going and you're guiding yourself through an experience. And then there's just the physical, like the sound, immersiveness you were just talking about, right? And so I think, immersive now, people are sort of applying it to their immersive work because it's a great word, right? It really encompasses a lot of possibilities, right? So. I'd be curious what others have to say about their immersive work.
Jelena: I think in a way it's not when you go to see the piece or to hear the piece, these pieces that are immersive, they are starting to see you in a way. It's like they grab you and they change the role. I'm remembering, for example, the installation Opera for a Small Room by Janet Cardiff and George Bures Miller. There is a cottage in a gallery normally, and then you as a kind of voyeur, you observe what happens there.
And there is a whole narrative that happens connected with all the LPs of the operas, etc. So you are not immersed in a technological way in that piece, but by that narrative and looking through the crack of the cottage in a way, you are really grabbed and installed in a way inside that that very cottage. So the piece is also immersive. Kamala: I think, to add to that, I think that that's one of the things that's exciting about doing music, music theater, opera as an immersive piece, which is letting people get closer to the sound. And very few people have been put next to an opera singer when they're singing. And it's something that's, that's pretty amazing, you know, to the physicality of it and the volume of it and the vibration in the body and all of these other things that you, you can't get when you're sitting far away from a proscenium stage.
So that, that to me is also very interesting is actual -- the physical experience of the sound. Ellen: And I want to take off that, with sort of what Kamala said and what everyone had said in that, when I work on a piece, I actually have in past pieces worked with scientists in human computer interaction. And the reason for that is I'm not trying to make a science piece, but I'm really working with setting up feedback loops, and feedback loops loop into agency. And they also loop into what was said about motion capture. And the VR experiences that work the best besides mixed reality,
are those in which the physical body of the person in motion, in the VR experience, is actually moving. And that's a whole new jump that's happened over the pandemic, as there are new motion capture suits for group interactivity. The type of human computer interaction via feedback loops that I work with have a lot to do with gaze, touch, motion and of course, biometric interactivity. In other words, what is immersive? And the immersive is the human senses being absorbed and changed and reacting and interacting to the environment and other sensory experience that creates a sort of unique feedback loop. And that's what I define as immersive.
There's some type of feedback loop between the digitization, the experience, and the audience and the performer. Jingyi: Yeah. I really like this idea of this natural feedback loop that engages fully with our five senses, not just the visual domain. And I think what what Yelena said just now about the medium of sound itself to create immersion is super insightful because if you think about it, you know, when it comes to sight, we can always close our eyes whenever we want, right? So we can choose what to look at or not look. But our ears, we just we can't shut our ears. We are constantly exposed to whatever sounds there are. So, in a way that kind of makes sound or music a more easier medium to get at this immersive state.
And also same goes with, you know, the sense of touch, right? You can't exactly be not like -- cut yourself away from your environment. So this idea of this feedback loop comes into play here. So that's really, really fascinating. Um, so now I'm trying to play the devil's advocate here.
So given all this hype, you know, publicity surrounding these newer technologies, could it be possible that we are getting so caught up with all this novelty that we've lost sight of artistic feasibility or this sort of like compatibility? Yeah. So who wants to take that on? Or maybe another way to ask this is, does it work all the time? Have you seen cases where it doesn't work as successfully as you thought it would in a performance event? Kamala: I think it depends on your definition of success. You know, some things are not meant to be linear narrative. Some things are meant to be abstract and poetic. Some things are purely experimental just to see what happens. And I think there's room for all of them. You know, I think this is the other thing that I hope that opera has learned from the pandemic is there are many kinds of opera. There are many ways to approach making it. And there, you know, there is an audience for all of them.
Right? So we should allow the ecosystem to be larger than it has been. Jingyi: Mm hmm. Jelena: I would maybe like to remember Philip Glass here. I interviewed him in 2007 in Lisbon, I think. And then we were talking about new media in opera, etc. And then Philip said, you know, when you will have, like 3-D projection in the room, then you'll have something by that.
Like, it's not really that. In a way I can share this this view, because what the pandemic also brought to me is this bitter taste of two dimensionality. We were all on the screens all the time, and that was just too much. And it provoked all kinds of fatigues and zoom fatigue at some point. And I was watching after all these blockades of theaters and operas, everything was shut down and wiped down by pandemic.
I still have this bitter feeling of how everything was just silent. For me, actually, it was not -- I still don't see it as something that really induced creativity. I more see it as an event that tried to repress the arts, still. Yeah. So in a way, the piece that I liked the most during the pandemic, in a way problematized all this two dimensionality of Zoom
and all these little screens that we were involved with, was actually Laurie Anderson's Spending the War Without You. This lecture performance combination of the commentary of her own life, of all kinds of thoughts about life and art and media also. That was something that for me was an interesting moment in the pandemic also to make the footnote. Because in a way it tried to go out of this two dimensionality of the screen that we were all linked to at the moment.
Ellen: Well, I want to say something about human capability that I've learned from working with performers in these experiences. And that's the question of thresholding. And thresholding in human cognition is a very underrated thing, because what I've learned, and I'm sure you see this when you train performers in endurance, is that human cognition can only take certain kind of mediated experiences for a certain amount of time, and then it thresholds or stops. And so when the question is, have you seen unsuccessful examples of this? The unsuccessful examples I've seen is where the creator of the event doesn't take into consideration the thresholds of the audience and tends to oversaturate, overwhelm and overdo it. And in that place, the audience -- like being on a zoom for like 5 hours is one example of that. You know, it's, it's -- you just can't take it.
And I think when you say what are the unsuccessful examples, for me that was the most -- that was the thing I noticed the most is that these flat, two dimensional experiences do not understand, for the most part, the thresholding of the person who is cognicizing or viewing them. And that for me was a big learning curve to understand that with these types of mediated experiences. Kamala: I would say that's true for real ones to. [laughter] Brian: I was going to sort of take it away from, from the zoom, you know, flat panel experience to, you know, when you see something that doesn't work in the context of the theater or, you know, site specific. And for me, it's usually it feels, it comes across as though the designers who are using all these different technologies, director, composer, librettist are not communicating.
Jingyi: Right. Brian: And so it's like this collision of ideas as opposed to a larger conversation about the concept, what are we doing together and how are these things interfacing? And when it's, when you do that, it can blow people's minds because then nothing's sticking out. You forget there's video, actually, because you're in it, or you forget there's these other kind of technologies you're using. So that's when I've seen it fail. It's just you can tell it's a lack of communication. Jingyi: And also probably because it's the first time opera companies or iindie opera companies are experimenting with these really new technologies that haven't been experimented before.
So they're still trying. So the technologies are kind of doing their thing. The composers are doing their thing. So maybe it's less about the lack of communication, but they haven't had the experience to work together before. So. So. So I guess only, you know, in the, in the near future, there will be much more productive exchange and more learning from each other in this whole, in this opera making process. I think that would be very good. Brian: Experience, yeah. Jingyi: Yeah, for sure.
Brian: It's just everybody learning from each other. Jingyi: Mm hmm. Kamala: And I would say also learning the new performance practices, because what works on a proscenium is not necessarily what's going to work in an immersive situation or in a situation with new technologies. And it's really hard for people to leave behind the techniques that they have spent so long getting good at to try something new. But that's what has to happen. Jingyi: Mm hmm.
Jelena: There is also one one thing that is very special and very important for every opera, I think, whether conventional opera or the contemporary progressive experimental piece, and I almost take it as a kind of diagnose of the piece sometimes, and that is the core of the opera, is the relationship between the singing voice and the body that sings in all kinds of modalities. So I think it's very important when you watch the new piece to sometimes in a way recognize that question and ask yourself, like, what happens with the score of the opera there? If it's constantly multiplied, either in a same kind of way, like conventional opera does, then I think those pieces have a tendency not to be very like progressive, or they just multiply the concepts that we saw or the -- for many times before. And sometimes even a close excursion in a piece that reinvents the relationship between the body and the voice, whether that is by the new media or by some playing with the gender roles or some de-synchronization between the body and the voice.
Then normally something progressive happens. Jingyi: Yeah. I think this is a really, really fascinating, you know, just looking at the relationship between voice and the body in cases where they are disembodied, right, singing happens, and the possibilities, the potentials for all that.
Um, so the next question is, I'm just really fascinated by all of you and all of the work that you're doing, especially all of you on this panel wears many hats, right? For instance, Jelena, you are an opera scholar, but also dramaturge, music critic, curator. Brian, you direct both operas and films and then collaborate closely with Philip Glass. Kamala, you're a composer that straddles, you know, boundary of experimental music, contemporary opera, and also vast and new technologies. Ellen, you are a new media artist, curator, critic, educator who is also very interested in the possibilities brought about by AI.
So my question for all of you is how does each of you navigate this multifaceted role that you play? Does each artistic role inform your approach or thinking in other roles? How so and in what ways? So anyone can start. Jelena: I think it's like when you cook in the kitchen, you know. Like, you can try the new recipe, then you're a researcher, you can try to perform a recipe that is already familiar, then you are a kind of performer. You can go to shop for nice ingredients. You can do all these roles, you can serve the dish or find a new way how to serve it.
So I see it more like that. And it's all cooking in a way. It's all the world of opera. So especially with contemporary opera, which does not have very strong institutions in terms of an opera building, of the system of stars and the whole protocol like conventional opera does, I think it's even necessary to do it in that kind of way because otherwise you you'll miss some dimension of that world. Jingyi: Mm hmm. Kamala: I guess I could also speak to the context of the United States in particular, where there isn't a lot of funding for the arts.
And so often if you want to make something experimental or that doesn't fit neatly into the boundaries of some established genre or funding structure, you have to do everything yourself because the infrastructure isn't there. And so, you know, that's something that you see a lot from people who are interested in you know, what is going to be something that pushes against the form is that they have to, they have to invent things and they have to invent new structures in order to make them happen. And then over time, maybe those structures become established like with the people from Bang On a Can for example, that when they started out, they were these sort of rebellious and downtown composers. And now they're very established and they have their own music festival that they do every year. Or even someone like Laurie Anderson, who is this amazing, innovative, creative person and just had a solo show at the Hirshhorn in D.C.,
which was fabulous, but a major solo retrospective at a giant institution. So I think there's this interesting thing about straddling both because because that's the way that it works. That's the path. Jingyi: Mm hmm.
Brian: And to riff off of that, you know, it's like you find a lot of these young companies, which I admire so much, who have no money whatsoever. And they're doing, they're reaching out to the community. And we're going to do an opera. We're going to talk to these people over here who have a puppet company. And we're going to collaborate and we're going to tell the story through the physicality of puppets while the singers dance behind them. The list is endless of what has been done, but it sort of speaks to what we're talking about here. And, you know, of just what means you have if people want to create, they will find a way.
And now we're we're doing it through technology, but it's just like throwing a wider net, you know, to make it happen. Ellen: Yeah. And, and something that's very relevant is community. And, you know, all of you are so deeply in -- soaked and such experts in opera and your base community is the operatic, even if it's the operatic indie or operatic indie music community.
And what's interesting about the pieces I make is the prime audience for these is actually the audience and then human computer interaction engineers, artificial intelligence people, computer scientists. I mean, there's, that's part of the people who I'm speaking to when I make these pieces, because my work always employs technologies that are breaking. And when they're breaking, right at the very beginning, Kamala, you said you did VR in 2016, and that's when I made my first brainwave opera. You know, it's, it's -- there is almost no precedents.
So the forms are messy. And the audience is, it's very hard for a traditional opera audience, in terms of community, to embrace this. They tend to denounce it because the tradition is so wonderful and so grand and so deep and so -- has a lot of longevity. And, you know, they're right. But these new forms break ground in a way which is very disruptive and breaks the definitions.
And so when you talk about how do you jump between these roles, it's like, who do I send this to? Who do you, who do you have in dialog? And I think that's also part of it. Like this is, what we're doing right now is both to the opera community and also to other communities who would be interested, who might be new media or might be more technical or might be, you know, more physically oriented. I don't know. So that's part of it is what communities are you speaking to besides the ticket paying audience? Brian: Ellen, Do you want me to play your Noor right now? Because it sort of makes sense to play that. Is that okay with everybody? Ellen: And that's the first one before I started using A.I. but you'll see the performer wearing a brainwave headset, triggering every sound you hear and every visual you hear, except for our talking to each other, is triggered by her brainwaves.
Brian: Cool. Here we go. VO: Inayat Khan was the daughter of Hazrat Inayat Khan, who brought Sufism to the West. >> It's a tradition in London and Paris. VO: He died when Noor was 13.
>> A man with a strong message. >> My irresistible. My father. VO: The family moved to France. World War Two broke out. >> Ariel bombardment. >> The world I knew changed all of a sudden. >> We're at war. >> We were looking for a new home. VO: But Noor was not content doing nothing, and she joined British secret intelligence, training as a wireless operator. Ellen: So, you know, the -- every, every -- the images were key to emotional states, yet were randomly generated.
And there were four emotions that came off her brainwaves, which were interest, excitement, frustration and meditation. And so the sounds were composed by a sonic composer. And if you see, she's touching the audience, she's looking at them, she's moving through them. And it took five months to train her to do that.
Without triggering the same, you know, without over triggering her brainwaves. And those brainwaves can't be fake. Like in acting, you can fake it. You can't fake your brainwaves. No, you just can't. Believe me, we tried. You can't. So that's what I mean by a feedback loop happening.
And there was no proscenium stage. Yet there was no invasiveness of the audience. It was very gentle and very soft and very inviting. No one felt threatened or invaded by their personal space or agency.
So I just wanted to emphasize that as well. Jingyi: Yeah. I'm really, I'm really amazed by how creative artists who don't work, work outside of the mainstream, you know, opera institutions, how they are the ones who are spearheading, you know, a lot of changes and excitement in the opera field today, precisely because I think they are really vested on, you know, the on the ground collaborations with different kinds of artists. So a lot is happening, right. And in recent years, there's also this,
you can see this transformation. Traditional opera companies where they are kind of learning from indie opera companies like greenscreen technology, you know what Yuval Sharon was doing, I just went to watch his The Valkyries at Detroit last week. Right. This idea of having these two layers of performance, one is the physical actors, they're performing and singing and then in real time being translated onto this digital projection with this virtual set. So, and that is something that was happening in the opera companies, and then he just took it up and then trying to, you know, make it more mainstream in traditional companies. So there is this, so I like this kind of dynamic creative exchange between -- in the opera ecology of today, I thought it was really, really, really fascinating. Brian: You know
what was interesting about that piece if I may because I was just researching a little bit doing -- I've done a ton of green screening for projects where you take the actor and you have them sing an Aria and then they can be singing an aria with themselves because you know, because you've done it before, right? And that's that's wonderful. What was interesting about that is kind of the transmogrification that he pulled off with a full green screen live, which is very Brechtian. And we're seeing how it's created, right? Then you have people with motion capture suits on, Ellen, on stage, so, and so you're -- and people in costumes, right? So all three of those kind of technologies are blended in the moment. Right. And then you have, it's sort of the, the kind of fully blended version being projected. Right. And that's just, it's a bit, I think that could be a bit exhausting for the audience
right, because they're going back and forth and it might pull them out. I'm not sure you'd want to see every opera like that. It was great experiment, really fun, but it's really to sort of help people also understand, I think, and get used to what is possible as we bring this to the bigger stages and the bigger pieces. So I thought that was really quite cool. Jingyi: Mm hmm.
Yeah. And what was kind of also interesting in that production was, you see those green men was trying to kind of flapping the capes of the gods in Valhalla and then just literally see them flapping the capes, which creates that magic on the, on the top level, you know, of the -- so there were a lot of audiences who were laughing at those sort of moments who were in agreement. Just come in and go out, come in and go and doing those kind of strange things on stage. Brian: Because they're invisible, right? I mean, that's what's so cool about it, right? Yeah. Something like that. Jingyi: It's cool. Magda: All right. Excellent. I want to jump in and just open the floor to our participants in case anybody has any questions. And I just want to kind of give you guys two observations coming from the conversations that I noticed. The first one is that it seems that the most successful pieces are the ones who sort of combine the content and the form.
So like, the form comments on the content and vice versa yeah? There is sort of acknowledgment that we are dealing with a new form that is trying to recreate preexisting works or recreate preexisting models of doing things onto new technologies. Acknowledge that this is a new technology and that this has a completely different way of doing things. And so the content of it is going to be commenting on that. And I really like that. In Kamala's work this happens a lot, yeah? With the little piece which we saw at the beginning, with the here's looking at you. It's very much also there because the technology is kind of the surveillance technology which audience members experience in the moment is part of the content of the theme of the show as well.
It seems to be that those works which engage and play with that relationship between form and content seem to be the most successful. The second observation that I sort of -- that came to me when I was listening to what you guys talk about was that there might be a kind of a new field opening in cognition and technologies and opera arts and cognition, yeah? I'm kind of thinking this on the spot right now, brainstorming with you, but I'm wondering if there is, if there is an opening for musicians and artists and technologists working with cognitive neuroscientists in some way and trying to figure out how those things actually affect us, whether in a positive or negative way, whether they have, can have some kind of medical application even. Yeah? And for the artists, you know and for a broader industry, music industry or performing arts industry, I really do try to really understand what kind of neurological impact those technologies make. I mean, we already had some of that, you know, with film, yeah? A lot of, you know, psychologists and neuroscientists kind of trying to figure out, you know in the 1960's and the 1970's how film, how people respond to film. What does film do to your psyche? And I feel like maybe there will be another kind of branch of research going in this direction, because it seems that it's a whole new area that we have not yet fully explored. And Ellen speaks so wonderfully and so, with so much insight on that.
That made me think, well, maybe there is a whole area that remains to be discovered. All right, so this is my two pennies. Ramona's going to jump in and do you want to give questions or do you want to comment on the panel? Ramona: I think, why don't we, I think -- I love your questions. We should have the responses first, then, otherwise, there's so many, so much already on the table. And then I'll jump in after that. Magda: Well, this was not, I mean, this wasn't really questions, was just me, you know, trying to summarize, um, summarize your points. But if anybody has a response to it, I would love to hear what you guys have to say.
Kamala: I guess I would just, I would say, you know, there is a branch of psychomusicology that is looking at a lot of these things. Like [a researcher] has some neural net models of how people experience pitch. But I think there is a lot of opportunity in that a lot of these studies don't actually use music. So. Or at least when I, I have a former life as a as a psycholinguist, so that's actually what my doctorate is in. And when I was working on my doctorate, there were not studies about this, but this was a while ago. So there's probably more out there. But I do think that there, you know,
one of the things that we always talked about in my lab was the fact that a lot of these studies use single pitches or sine waves rather than using music as the materials that are being analyzed for the brain. And that's not really the same. So I think there is a lot of opportunity there. Ellen: And I just want to thank you, Kamala, because I never knew. Who knew, right? That's interesting. I, I do work with human computer interaction scientists and I get into raging debates with them, actually.
And I'm, because you see, when they do these studies and they do do the studies, what they have is they have one question and they use the creative practice in the service of that question. And I have tremendous arguments with them because I say that the creative practice is not the servant of the scientists. And we've really gone at it some times. And so, yes, there is, there are medical applications.
That's primarily how the arts are integrated into the scientific community is through medical and therapeutic studies. And I take issue with that, um, strong issue with it, and there's a lot of flash points on it, but there's also a lot of breakthroughs in the ability to help people who suffered traumatic brain damage, vision loss, motion loss. I mean, there's been tremendous breakthroughs on the other hand, so it has a dual edge to it of breakthroughs and also subservience of the arts to the scientific breakthroughs. So it's a very strange tension that exists between the two communities. Ramona: I actually just wanted to jump in, and off of what you're saying, Ellen, and also in general, given the immense sort of interdisciplinarity and innovation that's in your work, and all of your works about this question of communication, because I've found that often, you know, as you're sort of working with computer scientists and VR specialists and -- that often also that communication and really kind of finding a way of collaborating together and speaking the same language and wanting the same things in rehearsal can be really quite complicated to get to. So I was wondering how you sort of address that and how that process works for all of you.
Kamala: I think the interesting thing about working with people from a different discipline is learning their language. And that every discipline has its jargon and often you'll be using the same words and you won't necessarily mean the same thing. So learning, learning jargon and learning also just norms of collaboration is really important. For example, when we were working on Looking at You, we were working with coders from Bandcamp.com and privacy researchers from Carnegie Mellon. And a lot of them, you know, they're coders.
So they're used to going off and working on their own and not talking to people while they're working on their project. And what ended up happening is one of the coders went and made a change to our database the week before the show opened and broke it because he didn't talk to anybody else about what they were doing. And so then we all had to have a conversation about how in this kind of a situation where we have to be communicative in a way that they don't always have to be when they're working on their projects on their own. So that was just a lesson in how the ways that people work are really specific to the discipline they're in. So having that conversation before they break your database is probably a good idea. Brian: Communication. Magda: Don't let your coder break your show the day before. After tech, nobody touches anything after tech.
I don't care how much tempted you are, yeah? Brian: Yeah. That's a real thing, though, right? We don't get to change anything once we've done the rehearsal. We have to just walk away. Magda: We just deal with it. Sorry. I know there is a bug, but it's working. Brian: Yeah. Yeah, those changes. Magda: Do we have any questions in the audience? Anybody want to jump in? Our audience members, attendees? Anybody. If you have a question, you can type it or you can let us know and we will let you speak. Anybody want to jump in? People are not too talkative today, but we can continue our conversation.
That's okay too. If anyone has any questions, please feel free to contribute. Jingyi, do you have any other thoughts that you wanted to bring forward? Jingyi: Yeah, sure. So I have also prepared some kind of speaker specific questions at the end. So this is like depending on how much time we have. So I think we have the time for this. So maybe while we were waiting for the audience questions coming in, we can, I can start asking those questions. So maybe we can start off same thing with, in alphabetical order, so starting with Jelena.
Because Jelena is also part of my edited book project. So I'm really fortunate that she could write about her latest research on landscape dramaturgy in opera, which kind of departs from this kind of logo centric thought and moving to kind of eccentric epistemologies and performance events. So could you share with us more on this new concept and how this new dramaturgical sensibility illuminates these contemporary operas that you are currently studying? You're muted. Jelena: Sorry.
Thank you for the question. The concept of landscape dramaturgy, I read about it from the theorist Ana Vujanović. It's not a basically new thing, but this theory takes it and reworks it in a way and reinvents theconcept. So basically landscape dramaturgy, she says, what it has to do with dramaturgy and with landscape, and if it's in the classical sense, that landscape deals with the nature and dramaturgy deals with the drama, then she says, you just do nothing with landscape and dramaturgy.
Basically what she does is she discusses the perspective, the perspective as a way of seeing things and a way of understanding things that is so deeply rooted in our culture that we even don't, we are not even conscious of it when we see some performances that are even, that developed with the concepts that, for example, does not have perspective or organization on stage, but the way how we perceive things still deals with this classical perspective. So she discusses the notion of perspective. She discusses the notion of of the landscape, mentioning the the concept of straying also, the concept of straying, like making the road that is not the straight road, but kind of wandering around. And then the concept of dramaturgy more in a sense, like Hans-Thies Lehmann defined post-dramatic theater.
Like when the verbal text is not the one that is in the hierarchy of the theatrical texts, the most important one. But all the other texts like the scenography, the costumes, the lights, verbal text also have kind of equal, equal or in a way disturbed role when compared with the classical way how it was done. So in that sense, what concept of landscape dramaturgy proposes often is multiplying the points of view, like not one point of view, like in classical perspective, but multiplying them. And then for example, this you can, you can encounter. It's quite a few recent operas. One that comes to mind is again Michel van der Aa's Upload. The piece that deals with --
I think you had a chance also to see it in New York probably. So the father and daughter, the father decides to be uploaded and then he stays only as a file in the computer. And then the way how the performance develops through various screens on stage that communicate one with another, forming different entities.
That is precisely, I think, this multiplying of points of view that is mentioned in landscape dramaturgy concept. That is, that's like one example. Jingyi: So, let's move on to my next question for Ellen, so I think you answered part of this question already in our very interdisciplinary, you know, messy conversations, just now, but maybe, I'm really kind of fascinated by your new project. I think which you haven't really talked much about just now because it's really kind of working at the cutting edge of technology. It's based on speech to visual AI and its relation to epigenetic, which might be a very unfamiliar term to a lot of us here today. So could you speak more about this amazing work and the technologies involved and how it's materialized on stage? Ellen: Sure. And I just want to add one thing. I did see Upload and I want to congratulate people from
the Netherlands on the financial support they gave Michel van der Aa to do such a fantastic production. I just thought I'd put that plug in. Yeah. What's happened in the past year is new artificial intelligence technologies have arisen which are creating text or speech to text to visuals.
And right now they take a minute or so to create, but they're going to get faster. And also they're going to turn into movies. So as you begin to, let's say, map out a dialog and talk about it, you could get a whole scenario set up just by speaking it. And that's being referred to as prompt engineering. In other words, your speech is the prompt and you're going to engineer the visual by what you say in speech.
For a year already I've been working on this and you know, the technology is so experimental that it, unless you have the servers -- a server farm behind you in order to produce 5 seconds of a video, it takes 2 hours to render it. And that's the, you know, so that's how rough it is to begin to make this. But how it relates to epigenetics, and I'll explain what epigenetics is very easily. Epigenetics is inherited trauma that affects your RDNA, that's passed through cultures of diaspora in ways that are not necessarily verbal and not necessarily digitized or able to be tagged.
And as we enter the digital world, these things are going to be forgotten, lost, and fall through the cracks. So I'm beginning to explore my own epigenetic experiences because I am from a culture of diaspora, and that's what -- I'm calling from Poland, where I'll be working on a project over the next year about my own culture of diaspora that is from this area called the Pale of Lithuania. That's t
2023-02-24