Exclusive: Inside Lucasfilm’s “Star Wars” VR Project

The ILMxLab expects to make many more VR experiences for “Star Wars” and other films, and for every VR platform.

Exclusive: Inside Lucasfilm’s “Star Wars” VR Project


Anyone who’s tried virtual reality with other people around knows there’s a moment when they have to let go of worrying about how silly they look wearing a big, bulky headset. That was no different for famed Star Wars and Star Trek director J.J. Abrams when he was examining early versions of Jakku Spy, an official Star Wars VR project that Lucasfilm released in the final days before the opening of The Force Awakens.

“Poor J.J., with everybody watching, (he had) to put the gear on and look around,” said Jeff White, a visual effects supervisor at Lucasfilm’s Industrial Light & Magic (ILM) who helped oversee the production of Jakku Spy. “But it was great. He really gave us some good direction, and changes to make, which really helped improve it.”

Lucasfilm released Jakku Spy, which puts viewers in the role of a Resistance agent in the desert world of Jakku, last month. But the company is only now, exclusively to Fast Company, opening up to talk about the process behind its first foray into an official Star Wars VR experience.

It surely won’t be Lucasfilm’s last, especially as high-end virtual reality gear like the Oculus Rift and HTC’s Vive hit the market this spring, enabling far richer VR experiences than even the episodic nine-part Jakku Spy, which was made specifically for Google’s available-to-almost-everyone Cardboard VR player.

Google said yesterday that it has shipped 5 million Cardboards, and that users have installed the Cardboard app more than 25 million times. Among the most desired versions of branded Cardboards, of course, is a limited edition R2-D2 model released by Google.


Projects like Jakku Spy could be very good news for the VR industry as more and more people get their hands on hardware ranging from the Rift and Vive to lower-end devices like Samsung’s Gear VR or Cardboard, and want more content to watch. Analysts predict that by 2020, the entire virtual reality ecosystem will generate $30 billion in annual revenue, while VR content alone is expected to be worth $5.4 billion a year by 2025.

Many Hollywood studios are getting into the VR business. Among those taking the biggest leap is 20th Century Fox, which made the 20-minute-long Martian VR Experience, which was based on the hit Matt Damon film and used visual assets from its production, but which was an interactive project that went well beyond simple marketing.

Lucasfilm, too, is relying on actual film assets for its VR projects, something which is a huge advantage when making a Star Wars virtual reality experience.

“That’s one of the great things about the way we’re authoring this content and the technology we’ve developed,” said Rob Bredow, the head of the ILMxLab, which produces immersive content for Lucasfilm. “We’re leveraging off of all the ILM experience, how we light, and shade, and create these materials and actually ingest them into the” final product.

Added Bredow, being able to access the same content used to make feature films “lets us work on a movie like [The Force Awakens] and then create virtual reality experiences using that same world.”


As Bredow explained it, Jakku Spy, which can be found on Apple’s App Store, Google Play, or Amazon, was only conceived after The Force Awakens was shot. But because the Star Wars visual effects team at ILM was taking 360-degree photos, and collecting plenty of other reference material while on set, they had everything they needed to incorporate all that authentic content from the film into Jakku Spy.

At the same time, Bredow said, ILMxLab worked with Lucasfilm’s story team from the earliest stages of making Jakku Spy, allowing it to have input on what would make for a good VR project.

Since ILM does visual effects work throughout Hollywood, that’s something we’ll likely see repeated for other film-based VR projects.

“A lot of studios now realize that they want to get this material specifically for VR experiences, so they’ve been reaching out to xLab much earlier in the process,” White said. “It’s great if you can go in with exactly what the VR concept (is) and shoot that while you’re on set. You have all the lighting, you have the cast, you have everything you need to do.”

Origin Story

If Jakku Spy has an origin story, it may be a day about a year ago when someone in the hallway at Google mentioned Cardboard and Star Wars in the same breath.


“I had helped start the Cardboard project, and I’m obsessed” with Star Wars, said Clay Bavor, Google’s new head of VR, who upon overhearing that idle conversation knew “we need to make this happen.”

Although Jakku Spy was produced by Lucasfilm, both Google and Verizon collaborated on the project. In Google’s case, that meant providing expertise–as well as technical tools and early looks at the latest versions of the hardware–on how to make projects for Cardboard.

Of course, giving Cardboard owners a Star Wars experience was sure to be great for the platform. Bavor knew VR and Star Wars were a perfect match.

“What’s familiar is the worlds and the characters,” Bavor told Fast Company. “What’s new is feeling like they’re in a new world. The VR experience is new. You see people experience VR for the first time, you see smiles, you see awe, and you see delight, because it’s experiencing something [familiar] in a new way.”

Bredow said ILMxLab knew many people had tried VR experiences, like New York Times journalism, on Cardboard, and the lab wanted to do something different. While his team didn’t want to make any individual part of Jakku Spy more than two minutes long, they were excited about the opportunity to “sit people in this world of [The Force Awakens]” and try something episodic.


And what better way to introduce many people–superfans and casual viewers alike–to Star Wars VR than to put them inside the Millennium Falcon and give them the ability to spend a little time alongside BB-8, the fan-favorite rolling droid from the new movie.


The trick, of course, was figuring out a way to make something compelling about Star Wars without giving anything away.

“The idea of making it tie into the opening of the film and lead up to it was really fun and really tricky, because we had to be very careful about what [got] out there,” White said. “We tried to look through the trailers and find scenes where people had seen it from one perspective,” and give them another.

For example, White continued, the most well-known image from the trailers was of the giant star destroyer that had crash landed in the desert.

In Jakku Spy, viewers see that destroyed ship and “now, you turn around, and ‘Oh, there’s [main character] Rey’s speeder going to work,’” White said.


That sense of playing on what people already know about the movie also worked for building up the story over the course of the nine short episodes.

“We were trying to connect the world together,” Bredow said. “If you’re in the [episode] where Rey’s speeder goes by, [you turn and see] smoke. That’s the crashed Tie-fighter that you see in the next episode.”

Better VR For Better Gear

This year, as has been well chronicled, is consumer VR’s coming out. Cardboard has been out for 19 months, and the mid-level Gear VR hit the market last fall. Higher-end and more expensive gear like the Oculus Rift, HTC Vive, and Sony PlayStation VR are all coming out in the next few months.

At ILMxLab, that presents the opportunity to create experiences for every level of hardware quality.

“Our goal is to be developing premium experiences [for] each of those” levels, Bredow said.


Being part of ILM, which does visual effects work for nearly every Hollywood studio (much of which is scheduled well ahead of time) is also a big advantage.

“As we did with [The Force Awakens], we’re looking for opportunities where we can legitimately complement the storytelling of the films going forward,” Bredow said. “And it’s great to have years of films laid out in advance so we can look and see how the technology is going to map to the storytelling opportunities, and when there’s going to be enough volume in the market to be able to address some of these high-end experiences [so we can] create for these high-end devices.”

Virtual Reality’s First Person Shooter Problem

About the author

Daniel Terdiman is a San Francisco-based technology journalist with nearly 20 years of experience. A veteran of CNET and VentureBeat, Daniel has also written for Wired, The New York Times, Time, and many other publications