Production & Design
ProcessAlgorithm’s Art Direction
As a starting point, we examined the collective materials from director Jo Mangan and the communities involved in the original workshops. Following this we participated in a number of workshops with INO, the first of which explored the creation of a narrative in 3D space using Unreal Engine, the platform that the Virtual Reality opera would be built upon.
The community participated in groups to build their own worlds, and shared them with each other to determine if the ideas were clear, resulting in a discussion as to what is necessary to tell an effective story in a virtual setting.
After these workshops we began a collaborative dialogue with Jo Mangan on the visual aesthetic and explored different image making approaches.
To achieve the surreal look of Out of the Ordinary, our team employed a combination of digital painting and Artificial Intelligence (AI) visualisation techniques. This is a technique where keywords and themes from the community workshops were used to generate AI art. This allowed for quick iteration and many different moods to be created in a short space of time. The opera features an array of dream-like landscapes, from an underwater paradise to a nightmarish apocalypse, which were imagined to accompany the opera.
For example, here you can see two very contrasting landcapes, the first titled 'underwater' the second 'riot':
We fleshed out these landscapes with textures and 3D objects which had been placed in the virtual space. The characters and creatures that populated these landscapes were purposely abstracted to expand the audience’s possible interpretation of the experience.
How this was implemented into the experience design
Working around limitations of the Oculus Quest 2, the VR headset that will be used to show the piece, was a challenge, which we addressed by creating a special pipeline which quickly generated images via AI. We developed 3D meshes from the textures based on this AI imagery. Throughout the arc of the story, each scene has its own unique palette and feel to distinguish it from the last, and enhance the feeling of progression.
Creating what felt like a natural environment in a very unnatural setting was a task in itself. Motion capture data recorded in the workshops was implemented in various actors in the scenes such as the fish in the underwater environment. The team additionally used many different particle systems, generated to replicate the randomness of nature.
Here you can see another two landscapes, titled ‘destroyed’ and 'snow':
In making the sound environments for this piece it was important to stay sonically true to the idea of an opera without breaking the immersion of the audience. We took some of the recorded audio elements from the opera and placed them in virtual space, using foley to further elaborate on the atmosphere of each different scene. Some of the elements were paired with animated visuals to support the narrative, and enhance Finola Merivale’s stellar compositions.