Disney CTO to speak about Disney’s work in XR as part of AIM XR series
Sean Corp, Content Strategist
It’s never a surprise when the Walt Disney Company breaks ground in the field of animation – they’ve been doing it since the company began in 1923. The Lion King, released in 2019, was hailed as a technical marvel in the field of photo-realistic animation, it was most groundbreaking in an unexpected field – virtual reality.
The film created an entirely new filmmaking process dubbed “virtual production” that could fundamentally change moviemaking, both animated and live-action films. The filmmakers built the landscapes and characters of The Lion King into a virtual world, and then the filmmakers donned VR tech to enter the world and decide how to film each scene of the movie. The “virtual production” process introduced old-school live-action filmmaking techniques into the most advanced animated production ever made.
This new way of filmmaking is just one space where Disney is using extended reality to support innovation and engagement in blockbuster movies, theme park attractions, online games, and toys.
U-M alumnus is Disney CTO
Jamie Voris, Chief Technology Officer at Walt Disney Studios and a University of Michigan alumnus of the School of Information, will speak about Disney’s work in the XR space as the final speaker of the Center for Academic Innovation’s AIM XR series. The event will take place at Forum Hall in Palmer Commons from 9:30-11:30 a.m. on March 27. Registration is required and space is limited.
With some of the world’s most well-known brands from Disney animation, Pixar, Marvel, and Star Wars, the brightest minds are working together at Disney to create compelling stories and delivering state-of-the-art experiences. Jamie will discuss Disney’s groundbreaking work, and for those looking to enter the always-changing XR field, he will talk about the kind of skills Disney is looking for as it builds out its XR output.
One of Disney’s latest developments was the launch of the 3,500-square-foot StudioLab in late 2018. StudioLab serves as an incubator for various technological breakthroughs the company is striving for, including work in XR on such films as the Lion King.
“One of the unique things about the culture that we have in technology here is that it is very egalitarian,” Voris said about working at Disney. “Anybody at any level can reach out, so everybody cooperates and collaborates.”
Breakthroughs in VR
Having already pioneered digital animation technology with The Jungle Book, director Jon Favreau was interested in seeing how virtual reality could change the process of creating animated movies.
And they did this by treating it more like a video game. In collaboration with MPC and Magnopus, Disney used the Unity game engine to build its virtual Lion King environments, essentially creating an open world for the filmmakers to explore and change.
As producer Jeffrey Silver put it: “We built a video game, the purpose of which is to make a movie. Inside that video game instead of cars and guns and points being scored, we got cameras and lights and lions.”
Utilizing a game engine allowed Disney’s production team immersion into the virtual environment and much easier collaboration as well as more real-time changes to the environment with every rock, tree, river, and shrub individually cataloged for easy addition or removal as needed.
Favreau wanted an animated film to feel imperfect and more like a nature documentary. He wanted the production process to feel more like a traditional film shoot with a collaborative nature the spontaneity of discovering new elements within planned sequences.
“The Human Element”
“All of this technology is used to harness the human element,” Favreau said. “The collaboration of humans is the most important part of this incredibly technical and technological film.”
To help bring traditional filmmaking techniques to animation, Favreau enlisted the help of Oscar-winning visual effects supervisor Robert Legato and Oscar-nominated cinematographer Caleb Deschanel.
It’s not common for high profile VFX artists and cinematographers to work on an animated film, Favreau, however, wanted them to help bring more traditional filmmaking techniques into the animated arena.
“Almost every movie you’ve ever loved, there is some shot in there that you like that was an accident,” Legato said.
That striving for imperfection and those happy accidents could only exist by exploring the film in virtual reality.
“Those surprises and those accidents that will sometimes impose themselves on a movie and make it better than what you imagined. We’re trying to create an imperfect world in which little things can happen that can create the kind of surprise we’re used to in live film,” said Deschanel, the Oscar-nominated cinematographer who lensed The Right Stuff and The Passion of the Christ.
Favreau and his lead crew, including Legato and Deschanel, would put on HTC Vive virtual reality headsets and be transported to various locations within the film and do a walkthrough just as they would on a traditional set. They could collaborate on how to frame and block the shots for a given sequence and work with a library of hundreds of thousands of assets to add or remove elements to provide the scene everything they would want.
There was also a virtual studio the filmmakers would utilize with traditional film equipment – lighting rigs, dolly tracks, Steadicams, and more with sensors on the equipment so they too could be transported into the virtual world.
From there, camera operators could place their camera in the virtual environment and frame their shot and film in “real time.”
Part of the goal was to capture the spontaneity and imperfections of the filmmaking process.
“The reason to do it the way we did,” Legato told FX Guide. “Is that that is how it has been done for a hundred years. You can’t improve on the filmmaking process.”
Favreau went on to utilize the same virtual production technique on his next high-profile Disney project, The Mandalorian. There, he replaced the typical previsualization process in the FX-heavy show with virtual reality as he worked with various directors and production heads on the episodes.
More about the AIM XR Series
AIM Extended Reality (XR) event series hosted by the Center for Academic Innovation that will explore how extended reality (XR) is being used in higher education and beyond. This speaker series stems from a Provost charge to engage in a new campus-wide XR Initiative. This initiative will formally ask us to consider how we can leverage emerging XR technologies to strengthen the quality of a Michigan education, cultivate an interdisciplinary scholarly community of practice at Michigan, and enhance a nationwide network for academic innovation. Learn more about the initiative on our XR initiative page.
XR at Michigan Summit
The Center for Academic Innovation is also hosting the XR at Michigan Summit on April 14 in Robertson Auditorium at the Ross School of Business. The all-day event will feature a keynote address by Jeremy Bailenson, U-M alumnus, Stanford professor and a leading voice in XR.
Bailenson is the founding director of Stanford University’s Virtual Human Interaction Lab. He studies the psychology of virtual and augmented reality. Speakers include a senior scientist from Microsoft Hololens, a former Snapchat executive, leaders of local XR startups, and U-M faculty and researchers that are teaching with and about XR technologies. There will be opportunities to try out augmented and virtual reality devices and experiences from sponsors Microsoft and Lenovo.
