RYOT’s Head of Immersive Jake Sally on the Challenges of Storytelling in Virtual Reality

Available to WrapPRO members

5 Questions: “The idea of free choice in VR is particularly tricky, because it’s never really your choice is it?,” Sally says

Robin Marchant / Getty

For creators of virtual-reality entertainments, the immersive storytelling format comes with new challenges — like keeping viewers focused on the plot despite being able to direct their attention anywhere. Or catering to a user’s desire for the so-called six degrees of freedom, the ability to move forward and backward, left and right, as well as up and down.

Verizon Media’s immersive company RYOT is well versed in these challenges. Founded in 2012 and acquired by Verizon in 2016, the company has worked on a growing number of projects and recently partnered with Paris-based emerging media company AtlasV to develop new immersive content like “Gloomy Eyes,” which premiered at the last month’s Sundance Film Festival.

RYOT’s head of immersive development, Jake Sally, shared his thoughts on the future of VR and extended reality (XR), which refers to all real-and-virtual combined environments.

1. One of the exciting things about VR is being able to consciously navigate and impact the story. How important is “free choice” when developing VR projects for RYOT?

The idea of free choice in VR is particularly tricky, because it’s never really your choice is it? Every element of the experience is being coded for the user. “Free choice” is really just the depth of interactivity that the VR creators provide for you — ranging from simply head moments to seemingly unique narratives depending on the characters you may choose to follow. There are a few exceptions, but very few are in VR.

For example, I love the concept of “No Man’s Sky,” a procedurally generated space adventure on PC — the idea of entire worlds being created by your decisions is a little ways off from being actualized for VR, but when it arrives I think people are going to develop a profound appreciation for VR that’s merely been hinted at until now.

2. How does VR impact the storytelling process from a creator’s perspective? With the ability for the viewer to look anywhere, does it become a challenge to keep them focused on the plot?

For as long as I’ve been working in the XR space, this question has plagued creators. What I’ve discovered is that for VR (and AR), the real challenge is to create an experience that’s so enthralling that the emotional core resonates regardless of the perspective that the viewer sees it through.

Our 6DOF VR projects “Mind at War” and “Vestige” accomplish this incredibly well — you’re placed inside the protagonist’s memoryscape and through voiceover learn about their difficult past. The emotion imbued within each protagonists’ words is what frames the viewer’s perspective, no matter where they’re looking.

For experiences that are more interactive, I think the question isn’t how to keep viewers focused on the plot, it’s really what aren’t you doing well that they want to focus on something other than the stories you’re telling them. You have the same freedom to look anywhere in a VR experience or in a movie theater, but no one goes into a movie theater and looks at their phone — unless they’re bored.

When viewers enter these dreamlike VR worlds, they want to be swept away — the only thing that will break that sense of wonder and discovery is poor UI/UX and story.

3. What innovations within the XR world are you most excited for this year?

I cannot wait for more voice-powered experiences. I’m an avid gamer so controllers are fairly intuitive for me in VR experiences, but for most, VR controllers are a barrier.

Allowing consumers to explore XR content with their voice is both intuitive and exciting — I think it’s one of the reasons why our AR experience, “Terminal 3,” was so successful despite the somewhat clunky HoloLens AR headset.

4. What does 5G mean for the progress of XR?

For years, XR content creators have been told they can’t create certain experiences because of the technical limitations of XR hardware and infrastructure streaming capabilities — 5G simply removes those limitations. XR content that normally would require content downloaded natively to a device (phone, computer, etc.) instead will be streamable — creating a seamless user experience.

a jester's tale ryot
“A Jester’s Tale” (RYOT)

5. What is your favorite XR project to come out of RYOT?

RYOT has a lot of innovative content coming down the pipeline, but the most immediate piece that I loved bringing to life is the AR experience, “A Jester’s Tale,” directed by Asad J. Malik.

Asad made an incredible piece last year, “Terminal 3,” which was a standout piece at Tribeca 2018 that led into a world tour. “T3” was made for an AR headset (the HoloLens) that no one had explored using for creative content and we knew that we’d hit a cultural nerve with impactful AR content. Off of that success, RYOT worked with Asad, developing “A Jester’s Tale” early last year when VR was hitting a cold spot and AR was beginning to heat up throughout the industry.

The success of “T3” showed us that people wanted emotionally engaging, thought-provoking AR content that went beyond any of the work we’d seen in the space to date and really highlighted the potential of the medium. So when Asad came up with the idea of a reverse-Turing test where you’re forced to prove your humanity to an AI, I knew it was the next project.

A dear friend of mine taught me to ask one simple question when evaluating XR projects — “How many miracles do we need to pull off for this to work?”

“A Jester’s Tale” required four miracles: We had to create the first independent Magic Leap project, somehow port extremely heavy volumetrically assets into the experience, make an exhibition set designed for live child actors and we had to do it all in six months.

Now back from Sundance, I can say we pulled it off — “AJT” not only opens people up to the untapped potential of narrative AR, but also did so while tying it back to the larger cultural discussion around AI in our society.