The Rhythm of Technology in the BMO Lab’s The Resistible Rise of Arturo Ui
Sebastian Samur
University of Toronto
Overview
This presentation offers a preliminary examination of how digital technologies used in the BMO Lab’s workshop production of Bertolt Brecht’s The Resistible Rise of Arturo Ui impact rhythm in performance. Rhythm is understood broadly here, not only as musical or metric rhythm, but on a more abstract level, which encompasses the overall rhythm of a performance (including its rehearsal process), rhythms in the performers’ bodies, as well as rhythms arising from work between all artists (director, actors, designers, etc).
The BMO Lab – affiliated with the University of Toronto’s Centre for Drama, Theatre and Performance Studies - uses research-creation to explore new digital technologies in performance, and their recent workshop production of Arturo Ui provides a useful case study for analysis, as it featured a range of technologies, including projections, sound, cameras, and, most notably, sensor technology worn by the actors, which will be the focus of this examination.
The Resistible Rise of Arturo Ui
As part of a residency with CanStage, the BMO Lab – led by David Rokeby - produced a workshop performance of Brecht’s The Resistible Rise of Arturo Ui, directed by Johanna Schall. One of the aims of this residency was to allow the lab to experiment with new technologies and how they might be applied in stage performance.
In Arturo Ui, voxel-based sensor technology was one of the main digital technologies explored, which provides data based on the sensor’s location in space. 17 sensors worn by the actor allow one to track the movement of the actor’s limbs, based on the measurements between them. They allow the artists and technical designers to play with activating sound, light and projection cues based on the actor’s movements. They can also allow the projection of a 3D avatar, based on the performer’s body. In Arturo Ui, both of these technical possibilities were played with.
The Sensor Technology
The sensor technology works through the activation of ‘trigger’ poses. When the performer takes a particular form or shape, it triggers the desired cue on stage. For example, one could program the lights to fade when the actor raises his arms, or a sound cue to play when he or she points. It is also possible to project a digital avatar on screen reflecting the relative positioning of the sensors. It is a different function of the same technology, which could be used to create a ‘digital double’ of the actor on stage, to use Steve Dixon’s concept (241). With appropriate software, the digital double could be made to appear as an uncanny version of the actor; however, it is also possible to play with the sensor’s output data so that a limb’s proportions or shape changes to something other than human. Moreover, it is even possible to create a digital mise-en-scene that could allow one actor to control the limbs of another actor’s avatar.
The Sensor Technology
In order to identify which of an actor’s positions should trigger a cue, the sensor technology uses Artificial Intelligence (AI) to train the system to recognize an exact pose. By repeating the desired pose multiple times in rehearsal, the artists can train the system to recognize the approximate limits of the pose. This allows for a degree of variability/flexibility for the artist, though they must still be somewhat precise in their movement.
The use of trigger poses with the technology will impact the overall choreography of a performance, as the artists must be careful to devise a scene in such a way that the actors do not risk unintentionally setting off a trigger pose. Overall, it is a manageable obstacle, but it does establish some limits to the movement choices in a scene as the artists must be conscious of repeated gestures, and each trigger pose should be relatively idiosyncratic. As the BMO Lab artists rehearsed Arturo Ui, they needed to be aware of the repertoire of trigger poses chosen for the performance as they shaped the choreography.
Example Demonstration
The BMO Lab provides a helpful example of how the technology is used during the opening scene of Arturo Ui here. Normally grey, the blue colour of the avatar at right indicates the actor has activated a trigger pose.
Rhythm in the Body
The production featured seasoned performers as well as students from the University of Toronto. Of note, two of the central performers – Sébastien Heins and Ryan Cunningham – had previously worked with the BMO Lab as artists-in-residence the year before and thus had a chance to better understand and incorporate the production’s use of sensor technology. In a personal interview, Heins credits this early period with allowing a smooth incorporation of the sensor technology:
“I found that at the beginning…there was a learning curve that was relatively steep but was made more… enjoyable because of the way that David Rokeby introduced all of those tools...He was able to very slowly introduce us to…first, what is a voxel? Then how do you trigger a voxel? Then what are the possibilities with sound with voxels? Then what are the possibilities with sound and lighting with voxels?...There’s not necessarily the space for that in a rehearsal process where you’re working towards production in three weeks.”
Rhythm in the Body
Already, then, Heins points to a major rhythmic shift in the rehearsal process of a production using the new digital technology – one that is often mirrored in other intermedial performances. There is a period of training – or dressage as Henri Lefebvre puts it in Rhythmanalysis – that aids the performer in incorporating the new technology through repetition. The performer must not only become accustomed to wearing the physical sensors themselves – essentially as a costume accessory, which can potentially limit particular movement choices – but must also learn how to work with all the immediate possibilities that the digital system can allow, such as triggering cues or projecting an avatar. Without this additional time, the performer’s work may be less organic, as they continue learning how to use it. Additional rehearsal time is also required to calibrate sensors at the beginning of performances, and to train the AI system to recognize a performance’s trigger poses.
While incorporating the sensor technology takes extra rehearsal time, it allows, in exchange, for immediacy on stage. Triggering cues with the technology can be instantaneous, providing a rapid response to a performer’s movements. The movements need not be overly broad or conspicuous, but can be more subtle (for example, the simple turn of a wrist, which was a gesture used in Arturo Ui), allowing for variation in performance and possibilities that mirror those of traditional stage performance.
Example Trigger Poses on Stage
Heins stretching his arms causes the spotlight to widen.
A flick of the wrist triggers stars to appear on screen.
Rhythm in the Body
In discussing the sensor technology, Heins describes it as a “creative limitation” rather than an obstacle to be overcome. The technology’s inherent logistical constraints challenge artists to figure out how best to incorporate it in performance. Do you hide the technology or make its capabilities conspicuous to the audience? When working correctly, Heins states that “the responsiveness of the system is exciting.” However, when it errs, he states: “There’s the funny feeling of impotence that comes when your suit – something about it – isn’t working.” Heins’ statement suggests a strong phenomenological connection between his body and the incorporated sensor suit (almost cyborg-like), evoking a personal sense of failure even if it is the technology that is at fault, not the actor. The feeling arguably affirms that Heins has organically incorporated the technology into his performance work to a high degree.
Rhythm in the Artistic Process
Heins’ statement also points to the important – yet often invisible - connection between the actor and off-stage technicians, which the technology can erase. In typical stage productions, it will be a stage manager or technician triggering cues based on the performer’s movements, rather than sensor technology (and indeed, in Arturo Ui, a stage manager was available to prompt cues in case the sensor suit failed). All artists working during a performance (on stage and off) must work together in synchrony for the production to succeed. Heins states that while “using this mocap technology you gain some independence and control and confidence and power, there’s a potential to lose the human connection between you and the stage manager.” The independence the sensor technology proffers can thus come at the cost of losing the positive sense of complicity that performers may develop with other artists in the production. Ultimately, it will be up to the director to determine what role technology will play in a production, which will impact the interrelations between all the artists.
Rhythm in the Artistic Process
The new technology also shifts working relations between the artists in other ways. The presence of the digital technician/designer throughout the rehearsal process (rather than closer to the performance opening night, which is more typical in traditional performance processes) is especially considerable, and likely a staple arrangement in most intermedial productions. When working on scenes that incorporate the sensor technology, there is often a three-way flow of communication between actor, director, and digital designer (rather than predominantly actor-director). When choreographing movement, for example, the artists will check to ensure that a given gesture feels organic for the actor, achieves the director’s desired vision for that moment, and concurrently functions to trigger a desired effect in the computer system. As Schall put it in interview, the AI is “another actor” on stage. The designer thus works as a proxy for the AI actor – a partner throughout the rehearsal period to ensure the AI system’s cues are properly learnt and the technology will respond appropriately to the actor on stage.
Rhythm in the Artistic Process
The technology working as another actor on stage becomes particularly explicit during an avatar chorus scene in Arturo Ui. During the “Whitewash Song,” Heins dances in front of a chorus of projected grey avatars who copy in synchrony Heins’ exact movements. Again, there is an interpersonal trade off with the technology. In a traditional performance incorporating a live chorus of actors, the audience would appreciate the work involved for the performers in order to synchronize their choreography as an ensemble. Instead, in Arturo Ui, the audience appreciates the technological spectacle of the uncanny chorus, rather than the virtuosity required to synchronize, which the technology all but guarantees (unless there is a glitch, as was the case during one performance wherein one arm sensor was unresponsive). The virtual chorus equally shifts the performer’s work as again there is a loss of interpersonal connection with other performers. Less rehearsal time is required, however, and the rehearsal process is different, as the actor initially performs his choreography while watching the avatars – much as some dancers or mimes might work in front of a mirror - to best understand how the avatars respond and appear. It is another way in which the new technology can significantly shift the artistic process.
Example of the Avatars Projected in Rehearsal
For additional technical explanation on the intermedial rehearsal process for the chorus scene, visit the BMO Lab’s page here.
Looking Ahead
Through the benefit of the CanStage residency, BMO Lab’s workshop production of The Resistible Rise of Arturo Ui provided a helpful avenue for experimenting with new digital technologies and assessing their potential uses in stage performance. Following the presentation, it remains an open question as to whether the technology’s instantaneity warrants replacing traditional sound and light cueing that would normally be conducted by a stage manager and/or technicians. As mentioned, there is the loss of artistic complicity, but there are also the logistical requirements of the tech system, as well as additional rehearsal time needed, which are not necessarily justified if simply replicating effects that can already be achieved using current creative process models. Instead, it may make more sense to highlight the technology for the audience – as with the use of the avatar chorus – and continue exploring the new creative outputs that can arise by making the technological ‘actor’ more visible on stage. New forms of performance training will inevitably emerge as actors learn to train the AI system to recognize trigger poses and to work in synchrony with their avatars. The intermedial work also demands shifts in artistic process, expanding beyond the actor-director binary, to include the technical designer - and ultimately the digital partner. Overall, then, it is clear the new technology’s rhythmic impact on performance is substantial, altering both the actor’s individual corporeal technique, as well as the creative processes involved in interacting with other production artists.
References