Capturing Performance - Part 1
ProcessVR holds many possibilities for the performing arts, not least, the possibility to capture real life movements and translate them to the virtual world.
During early development for Out of the Ordinary / As an nGnách, we spent a day with Virtual Reality Ireland, director Jo Mangan and mezzo-soprano Naomi Louisa O’Connell to explore the potential it holds for capturing opera performance in immersive media.
Using the micro opera TOUCH (an opera created specifically for film composed by Karen Power for INO’s #20ShotsofOpera project in 2020) as a test case, we created a short, immersive, animated version of the opera using Rokoko motion capture technology and Unreal. The performance was captured and created by Aisling Phelan and the team at Virtual Reality Ireland.
Initially, the 2D projection graphics from TOUCH, designed by Luca Truffarelli, were imported into the Unreal engine and the team at VRI worked with the video content to create video textures in Unreal for the immersive test. A number of pre-existing avatars were selected to layer over the motion capture data we would create, offering character options for the animation. The recorded audio track from TOUCH was used as a soundtrack and we arranged a playback track for the motion capture so the movements would reflect the original performance as closely as possible.
Naomi’s body movements were captured via a Rokoko motion capture technology. Jo directed Naomi for the physical performance aspects, along to the playback of the music. There was a spirit of experimentation and play to the session, working with happy accidents like Naomi’s avatar slipping through the plain of the virtual floor, essentially disappearing from the virtual scene, an interesting example of the opportunities that artists can find in what otherwise might be seen as glitches or problems in the technology.
Here's an example of how the motion capture and mapping all came together at the end of the day’s session.
The next step in our exploration of motion capture was to bring this activity to our Out of the Ordinary / As an nGnách co-creators through workshops held both in Tallaght and on the island of Inis Meáin. The movements we captured with participants will inform the animation of certain objects in the virtual world, be they jellyfish, fish, seaweed or birds. This allows us to deepen the involvement of participants in the performative elements of the opera, alongside more traditional performance opportunities such as singing or musical performance. Another step in the exciting journey of creating opera in the virtual world.