Death and the Powers

Death and the Powers is a robotic opera by Tod Machover of the MIT Media Lab commissioned and premiered for the Prince of Monaco. The opera features a Greek chorus of twelve 6-foot tall Operabots. The Walls (three 14-foot tall periaktoi) with giant displays glide around the stage as the set. I designed and built software to choreograph and puppeteer these robots during the show. This work served as my thesis for my Master’s of Engineer in Electrical Engineering and Computer Science at MIT. I also wrote audio processing plug-ins to drive a massive, 100+ speaker surround sound system used during the show and a 64 speaker linear array that created virtual soundfields via constructive and destructive interference.

MIT Media Lab’s robots are slick and expressive, visually evoking everything from iPods to 2001’s HAL, while remaining something entirely new.

TimeOut Boston

Tod was selected as a finalist for the 2012 Pulitzer Prize in Music for his score for Death and the Powers. The music is a beautiful mix of electronic and acoustic instruments with incredible sound design bolstered by the unprecedented surround sound setup. Alex McDowell designed all the outstanding visuals for the show, including the Operabots and the Walls. Karole Armitage did an amazing job choreographing the robots, a completely new medium! Robert Pinsky, US Poet Laureate, wrote the witty and insightful libretto. A few quick specs:

• three omni-wheels allow driving in any direction without turning
• 13 individually addressable channels of LED lighting
• head tilts for expressivity and elevates, growing the Operabot from 4′ to over 6′ tall
• powered by an OLPC (One Laptop Per Child) running Debian linux
• puppeteered by six robot operators using Xbox 360 controllers over 802.11n wireless
• motors and lights controlled by two Arduino Megas
• lights and head can animate in response to singers’ vocals, giving the illusion of the robot singing
• powered by a couple car batteries

I spent most of my time building the show control systems for the Operabots. Initially we were aiming for an entirely autonomous robot troupe but quickly found that with the precision required, this was infeasible. Instead, we settled on having six operators to drive the nine robots required by the staging. We had an extra three robots as backup in case one of the main bots failed. Each operator was situated behind the stage, up high in the wings. Each was equipped with his or her own computer with a touchscreen and an Xbox 360 video game controller. The monitor showed a live camera feed of a birds-eye-view of the stage and allowed the operators to select which robot they wanted to drive.

The system needed to be robust enough to run through show after show without failing. As well, the robots zip around at 15+ mph and weight 200 pounds! Safety was a real concern. The giant Walls weighed a couple tons but they moved slowly and had a tether with a kill switch to cut the power. On the other hand, the Operabots were entirely wireless and nimble. We put in failsafes through out the systems to protect the actors and the musicians in the pit. If communication was ever lost for even a second with the master computer, one of the puppeteers’ computers, the motor controllers, or the computer on board the robot, the motors would turn off.

Some screenshots of an early version of the show control software: