Crossover: Multi-threaded immersive theater experience

Pitch

Crossover is a multi-threaded narrative with 5 characters, 4 of whom are ghosts, and three rooms to choose from. Each room has a theme of the loss of relationship, the loss of children, and the loss of parents. The five characters are moving to different rooms, and you can either stay in a room or follow a character and move from scene to scene at any point. There's three acts, and the final scene is a grief ritual where each ghost has to deal with unresolved grief in order to cross over to the other side.

Crossover inspired by immersive theater experiences like Sleep No More, and we think that VR is a perfect medium where every character has their own story and you can choose who to follow and where to go. We think we've created a unique cinematic VR experience that goes beyond the type of on-rails storytelling that's been done so far.

Innovations

User Interface for Comfortable Locomotion between Cinematic VR scenes

We've come up with an effective and intuitive user interface for navigating between parallel story lines. Looking down creates a change blindness effect that results in a comfortable experience of moving from scene to scene. This type of scene navigation approach could be used within a live-action, 360-degree video and opens up the possibilities for creating a complicated and looping narrative structure that really only works well within VR and immersive theater experiences.

Multi-threaded narrative

This is an approach to VR storytelling that introduces agency to the user while still allowing the storyteller have control over creating a narrative puzzle. We hope to create a compelling & unique VR experience where you have to piece together all of the narrative fragments in order to see the big picture. The viewer can only experience 1/3 of the first two acts, which inspires either a repeat viewing or an engaging discussion with other people after they see it within a group context.

Human emotion and human faces as keys to coherence & presence

Some of the latest research into presence shows that immersion and cohesion are two keys components. A lot of focus has been placed upon the Place Illusion of being in another place, but we wanted to address the Plausibility Illusion by creating a coherent world where there are social interactions, human faces and emotions that create a believable world. We chose a low-fidelity graphic style and used Faceshift to do facial capture of the performances.

3D animation Editing Pipeline

We used a single depth camera to shoot each performance individually, which meant that we had to do a lot of editing. There aren't a lot of great tools available to do non-linear editing of multiple story lines, and so we ended up editing the audio in Adobe Premiere, copying the frame numbers of the cuts into an Excel spreadsheet, which then automatically generated Maya MEL scripts to be able to make the edits to the FBX files. There's still quite a lot of editing and data massaging that could be done, but this is a proof of concept that it's possible (though not easy or fun).

Grief Ritual

The story is a very personal story of the loss of my wife to suicide seven years after her father committed suicide. I've learned that grief is complex emotion where it can be soothing to hear about other people's stories of loss. This has been a process of creating the grief ritual that I wish I had available to me as I started on this journey. We believe that VR is a particularly well-suited to share the types of stories that can cultivate a sense of empathy and share personal stories that don't work as well in a 2D medium.

How it works

There is a map with three different rooms. Simply gaze and tap select a room to begin, and then at any point during the experience you can change to another room.There is either a dialogue or monologue happening in each scene that's happening in parallel and so you're always missing out on two other scenes. There are fixed camera positions to minimize complications with locomotion, but the user can still look around and choose which room to watch.

Built With

• faceshift
• unity