You’re tapping through your friends’ Snapchat stories, and suddenly you see something unexpected: Unlike the standard selfies with dancing butterflies or mouse ears, you see a video of your friend biking down the beach while an animated phoenix darts around in front of them, with glittery particles rushing toward you.
This is more than your typical face filter, which maps cute features and animations to your facial features and have been around since 2015. The phoenix, while clearly not real, looks like it’s actually at the beach with your friend. Unlike an animated sticker, where the phoenix might fly around without regard to its backdrop, the bird’s motions and the dazzling particles it emits are all contextual, getting larger as they get closer to the camera.
[Image: Snap Inc.]The phoenix’s magic trick is only possible through Snap’s new Spectacles 3, for which Snap is opening pre-orders today. Unlike previous versions, the Spectacles 3 have two cameras—one on each side of the wearer’s face. By putting another camera on its video-recording glasses, Snap’s engineers are able to calculate the distance between the wearer and the objects that they’re looking at, creating what Snap calls a “depth map.” Then, when users go to upload their Spectacles content to Snapchat, they can add a whole series of new effects that take advantage of this newfound understanding of the physical world.
It’s a new approach to Spectacles, distinct from Snap’s first foray into hardware back in 2016. The first Spectacles were brightly colored glasses that had a single embedded camera, enabling users to take 10-second clips of their lives, from their point of view. But while Spectacles seemed like they could be a breakthrough product for the company, which IPO’d just months after their launch, neither the first launch of Spectacles nor the second iteration in 2018 managed to convince millions to put cameras on their faces.
Perhaps that’s why for Spectacles 3, the company is keeping its expectations low. “We’re putting it out there with the goal of learning about some critical things,” a Snap spokesperson said. “We’re not trying to sell a billion of these.”
[Image: Snap Inc.]Instead, Snap is thinking of the new Spectacles as a test run in the company’s push to bring augmented reality to the masses—a trajectory the company began with its popular face filters. By putting another camera on Spectacles 3, Snap opens up a new realm of AR effects that users can add to their videos and images. It’s not just phoenixes; users can also add colorful filters that morph throughout a video based on how far away objects are in the shot and animated hearts that float around a video but burst when they come into contact with a real-world object. Snap is also opening up the depth map’s features to content creators, enabling to make their own AR filters through its DIY filter maker, Lens Studio.
It’s also priced for a limited audience. At $380 for a pair, Snap isn’t going after the casual consumer (previous versions cost $150 and $200). The Spectacles 3 are deliberately designed for early adopters who want to get their hands on the latest filters—and who can help Snap test out this step forward toward its vision of augmented reality. On top of applying AR filters to Spectacles videos, content creators will also be able to upload content straight to YouTube VR, making Spectacles a tool for creating three-dimensional video. Viewers can use a mobile VR headset to feel like they’re immersed in this type of video. Even photos taken with Spectacles 3 will have an added bonus: By stringing together the images from the left and right cameras, they’ll be instantly GIFable, in a single shot.
[Image: Snap Inc.]To understand the depth of the world around Spectacles wearers, Snap’s engineers utilize the same technique that your eyes do, called stereo vision. Our eyes feed two different images to our brains, and our brains calculate the differences between the images. Objects that are closer to us will look mostly the same in the two images, since our eyes tend to point toward your nose to focus on something up close. Objects that are farther away will look different, since our eyes tend to point parallel to each other. With two cameras, the Spectacles 3 can send both of these two images to your phone for processing—for creating depth maps of videos, Spectacles uploads the two videos to the cloud. Snap uses the difference between the same point in each image to determine the distance between that point and the camera. The 3D effects can then be added on the phone at a later time.
Perhaps the most intriguing piece of Spectacles 3 is an accessory that will come with each pair when Snap starts selling them in November: a cardboard headset that looks almost identical to cardboard VR headsets. But this isn’t designed to watch 3D videos from YouTube. Spectacles users can put their Snapchat app in a special stereoscopic mode, and then place their smartphone inside the headset to relive their own memories, from their own point of view.
[Image: Snap Inc.]Will anyone actually use this 3D viewer? Maybe not, since we’re all more accustomed to reliving memories through our much more convenient screens. Even avid Snapchatters may find that new, more realistic filters are not enough of a lure to actively use Spectacles 3 on a monthly basis—something that Snap has reportedly struggled with in previous iterations of the product. But even if few people actually use Spectacles, it feels like a step forward for Snap’s augmented reality ambitions because it enables Snap to use depth to make more contextual animations and lenses for the first time.
Though Spectacles have mostly been a commercial flop, Snap says that the glasses are part of its long game. “We have this long-term vision of computing overlays onto the world, and this idea that applications move from 2D screens out into the spatial and 3D space,” Snap CTO and cofounder Bobby Murphy told my colleague Mark Wilson a few weeks ago. “We know that hardware is going to be a major component of realizing that future, and so Spectacles, and our Snap Labs team that builds Spectacles, is a big part of our goal. The different iterations of Spectacles are just kind of pushing us along this path, and allowing us to explore and work on a hardware device from the ground up in a way that will allow us to create some really interesting experiences and interfaces for users in the future.”
Next up? According to a Snap spokesperson, Spectacles are moving closer and closer to bona fide AR glasses that can project images into your eyes. When that day arrives, you might be able to project phoenixes and bursting hearts into your eyes IRL, so you don’t have to wait until they’re uploaded to Snapchat to see them.
Recognize your company's culture of innovation by applying to this year's Best Workplaces for Innovators Awards before the extended deadline, April 12.
