Cradling her 1-year-old baby girl, Reese, in the middle of a large, high-ceilinged soundstage draped in green fabric, a young mother named Ashley Martin Scott may literally be the face of virtual reality’s future.
Scott and Reese are here to work on what in some ways is a VR interpretation of the groundbreaking Up documentary series, which has chronicled the lives of a group of Brits every seven years since 1964, when they were just 7 years old. It’s also a demo of a new VR filmmaking technique known as volumetric capture that many say is poised to eventually surpass 360-, or spherical video, the method most commonly used today.
Here in Culver City, California, a small film-industry company town in the middle of Los Angeles, Scott is fighting back tears as she speaks to a future Reese, with more than 40 off-the-shelf cameras arrayed around the room recording her every word and movement.
“Hi, Reese, it’s your mommy,” Scott begins. “It’s been the best year ever . . . watching you grow and learn.”
Not everything about having a young daughter is great. She’s terrified, in fact, thinking about Reese as a teenager going out on dates with boys. “There’s literally nothing in the world that scares me more,” Scott tells the dozens of cameras. But, “we’re going to get through it.”
Virtual reality technology has been around for decades, but it’s only over the last couple of years that it’s being targeted to a large number of users. That’s especially true now that consumer VR hardware is finally coming to market, first with Samsung’s Gear VR, which went on sale last month, and soon with the Oculus Rift, the HTC Vive, and Sony’s PlayStation VR, all of which are expected to be released in 2016. Virtual reality content is predicted to be worth $5.4 billion annually by 2025.
Computer-generated games still make up much of the content being created for virtual reality systems, but there’s a growing amount of live-action, or mixed live-action and CG content known as cinematic VR.
That type of content is still fairly new, so it’s hard to say there’s a traditional way of producing it. But if the film starring Scott and Reese was being shot the way most people would do it today, there would be a single multi-lens camera rig in the center of the scene, shooting in 360 degrees, and viewers watching later would be dropped into the middle of that scene, in a fixed position, and able to see all around them by turning their head.
That can certainly be compelling, and there are some fantastic examples of spherical VR available today, often involving action sports or music concerts. But to the founders of 8i, the startup hosting Scott and Reese’s shoot, it’s not nearly compelling enough. They think true realism–the cornerstone of VR–will come from viewers being able to actually move around in 3-D virtual spaces, interacting with 3-D video of real people, and choosing the angle–the shot as it were–that they want to watch.
“The viewer is the director. You choose your own shot,” says 8i cofounder Linc Gasking. “The director chooses the environment and the people and their actions, but doesn’t choose the shot, because that’s left to the viewer.”
A lot of experts agree: Volumetric VR, as 8i’s approach is known, may very well be virtual reality’s future.
“I love 8i, oh my God, I love 8i,” says Nonny de la Pena, a pioneer and cofounder of the Emblematic Group, which creates VR experiences. “I really think they are offering the ability to marry what’s kind of been a holy grail, photo-realism we can capture, and the computer graphics that allow for fully embodied, walk-around experiences.”
Investors think so too. To date, 8i, which Gasking cofounded with Eugene d’Eon in 2014, has raised $14.8 million in venture capital to build a media platform around volumetric VR. To do so, it has hired team members with backgrounds at companies like Weta Digital, YouTube, NVIDIA, Google, Pixar, Microsoft Research, Digital Domain, ILM, and others.
Scott Nolan, a partner at Founders Fund, which was an early investor in Oculus VR–acquired by Facebook for $2 billion in 2014–and which invested in 8i, said that while much of virtual reality hardware is becoming commoditized, 8i’s technology offered exactly the kind of immersive breakthroughs the firm was looking for.
“Virtual reality is meant to be immersive, to make you feel like you’re really there,” said Nolan. “For us, that means volumetric content . . . If there was a way to capture real video of real things in a volumetric environment, to interact with and experience those things, we thought that would be really valuable. We met 8i, and that’s exactly what they’re doing.”
Others have noticed, too, including the Sundance Film Festival, where 8i’s project, #100humans, featuring Scott and Reese, will be shown for the first time next month.
“With spherical video, you’re placed in the middle of somewhere, like a concert . . . looking around, like in a fishbowl,” says Gasking. “We wanted something more attuned to telling stories, [establishing] presence so you’re actually there, and there’s a more emotional connection.”
Adds Gasking, “The ability to create a human narrative for storytellers comes to a fore in volumetric. In 360 [spherical video], you’re stuck with the location . . . Everything’s fixed. With volumetric, everything’s possible.”
De la Pena agrees. “What 8i is doing is opening the doors to the future,” she says. “360 video can be very beautiful, but it’s also an adaptation of technology from the past. What 8i is doing is absolutely, fundamentally innovative.”
The emotional component of volumetric capture is key. Take Scott’s semi-annual visits with Reese to 8i, for example. Strapping on a VR headset, Scott can look at the video in the future and be transported back to that moment.
“You’re not just remembering it,” Gasking said. “You’re reliving it.”
In fact, he explained, when Scott got a chance to try out the video, she actually stepped into her body, he said. “She backed herself into her body. She put her hands up . . . and she felt like she was holding her baby again.”
By using 8i’s volumetric capture system, filmmakers can shoot VR content and then mix it with any kind of computer-generated environment they want. That’s because 8i’s software is designed to make it possible to extract the three-dimensional content and integrate it into other live-action or computer-generated environments.
That means, the company says, that volumetric capture would be ideal for dating and travel sites, or fashion businesses.
Or, say, someone could create a musical experience where users get to sit right next to the performer. “Instead of a concert,” Gasking posits, “you’re there inside your living room with Beyonce having a one-on-one experience.”
Although it’s best experienced on high-end VR rigs like a Vive or an Oculus Rift, 8i’s content can be viewed on any virtual reality device, and even on the web. At the same time, while all the content currently available for the platform is being shot by the company itself, it has recently opened its Culver City studio to outside content creators, and next year, it will begin allowing filmmakers to shoot volumetric content in their own studios, using 8i’s tools to process and distribute it.
In the future, it could be even easier. Gasking says that someday, users might be able to pull out their mobile phones and shoot volumetric video.
“The big dream,” he says, “is that a family could combine video from a bunch of cell phones.”
That’s still quite a long way off. But what’s possible with the technology today still impresses experts.
“We’re super-strong advocates” of volumetric capture, says Jeremy Selan, an engineer at Valve, which helped develop the software that powers HTC’s Vive.
The Vive, like the Oculus Rift, uses a positional tracking system that knows where a user is in physical space. That makes it possible for a user to move around in a virtual reality space, enabling a wider variety of VR experiences than are possible on systems like the Gear VR that don’t offer positional tracking.
Volumetric VR is one of the technologies that most takes advantage of that kind of tracking, since it can very closely follow where a user is in 3-D space.
As users “start purchasing higher-end VR systems, they’re going to develop a taste for what good positional tracking feels like,” Selan says, “and any content that doesn’t take advantage of that will just not be as compelling.”
Ted Schilowitz, the futurist at Twentieth Century Fox and an 8i adviser, said he thinks that there’s currently room in the VR industry for both spherical and volumetric capture. But that won’t be true for long.
“Over time, the medium will start to demand volumetric capture,” Schilowitz says. “Spherical, non-volumetric capture, which has tremendous value, doesn’t solve enough of the need for the medium. It’s more the short-term solution. As computing power gets better, and [VR headsets] get better, as everything gets locked in, users will demand that their devices offer full capabilities.”
With spherical video, he adds, users can’t look around objects in the way they can with volumetric. And that’s crucial for bringing in the “reality” part of VR.
“With virtual reality, the goal is to make the belief structure so real,” Schilowitz says, “that you can’t tell if something is really happening or not.”