The view from behind the lenses of a Magic Leap 2 is deceptive. What’s seen in this augmented reality headset floats in the middle distance, clear as a computer screen, but is not actually there. Looking around a room during a recent demonstration of the device, the digital overlay stands still, a glowing hologram of information that, despite appearances, is mere millimeters from the eye. The urge to reach out and try to grab it is almost impossible to resist.
Augmented reality offers a different way of interacting with the digital world, putting three-dimensional information or images or entire universes in front of the eyes while keeping the real world as a grounding base layer. Magic Leap 2, the newly released augmented reality headset from Magic Leap, a company founded in 2010 that quickly raised $2.6 billion in funding, is a major bet on the technology, which has both an alluring newness and an uncertain utility.
This was the perception of the company’s original device, the Magic Leap 1, when it was released in 2018 at a base price of $2,295. Though it was marketed as a consumer device, consumers seemed uninterested in the emerging concept of mixed or augmented reality.
Early reviewers were largely unimpressed with the image quality, and saw it more as an experimental interface geared toward fledgling AR developers than casual users. The company had hoped to sell 1 million units in its first year. It reportedly sold just 6,000 in its first six months.
Since then, augmented reality has become better understood, with products like Microsoft’s HoloLens securing a contract with the U.S. Department of Defense worth nearly $22 billion; the productivity-focused Lenovo ThinkReality A3; and even the recently released Meta Quest Pro from the company formerly known as Facebook aimed at providing a wearable window into the still questionably useful metaverse.
The hype and bust of the Magic Leap 1 led the company to rethink its strategy. Pivoting away from the prospect of the device replacing phones, televisions, and laptops, as founder and former CEO Rony Abovitz once suggested, the company narrowed in on the types of users who would see augmented reality as a tool. (Abovitz contends that the company hasn’t moved away from general consumers, and that the initial launch of Magic Leap 1 was intentionally focused on developers before broadening out to a wider user base. “We knew that professional and enterprise use cases were going to come before wide consumer use,” Abovitz tells Fast Company. “The main change Magic Leap made in the spring of 2020, following the pandemic and global economy crashing, was to shift from being a dual track company, enterprise and consumer, to a single track company.”)
Enterprise users are that single track focus for Magic Leap 2, from surgeons referencing an MRI in their peripheral vision to fire chiefs plotting out lines of attack on an active wildfire.
Creating a headset capable of meeting these demands required a rethink of what the company had built the first time around. A second version of the device would need to increase its power, its usability, and its overall relevance to industries more interested in solving problems and running businesses than playing games.
To explain how those challenges are being met, Magic Leap provided Fast Company an exclusive behind-the-scenes look at the design and development of Magic Leap 2, showing early prototypes and breaking down the metrics that reshaped the device for a more serious market. It’s an insider’s view of how an emerging technology gets made, and how its makers are betting that they can bring it into the mainstream.
A delicate balance
A circular problem lies at the core of the Magic Leap 2’s design, according to the company’s chief design officer, James Temple. To appeal to the intended market of enterprise users who would be wearing the device as part of a job, the device needed to offer as big a field of view as possible.
To offer that FOV, the device would need a more powerful, and heavier, computing system. But to be comfortable for enterprise users like surgeons and factory workers wearing it for hours at a time, the device needed to be light.
“What you’ve got to balance is the right amount of compute while being a wearable form factor, while giving you the FOV so you’re not perceived to have boundaries to your digital contents in relationship to the real world,” Temple says.
A bigger field of view would mean a heavier device, which in turn would mean a less comfortable device. A more comfortable fit would compromise the view and therefore the utility. Optimizing for one factor inevitably throws the others out of whack.
Striking a balance in a single one-size-fits-all device has taken nearly four years of development. The final product, released in late September at a starting price of $3,299, tries to solve for each of these interrelated problems.
Putting the device on—sliding it over the top of the head like a pair of goggles, tucking the cord-attached compute pack into a pants pocket, and grabbing the wireless handset—is a straightforward process. The weight of the headset, which the company clocks at a little more than half a pound, is about the same as a bike helmet.
Common virtual reality headsets on the market tend to be much bulkier, like a helmet with a brick of computer in front of the eyes. Sony’s Playstation VR headset, for example, weighs 600 grams, more than twice the 260 grams of the Magic Leap 2. Resting on the nose, forehead, and sides of the head, and subtly offset by a thin overhead strap, the weight is noticeable but not burdensome, at least in a brief demonstration.
For Magic Leap’s designers, getting the thing onto people’s heads proved to be a surprisingly complex challenge. It was also a notable hitch in the first-generation device that prompted significant user feedback, both from doctors in the operating room and workers in factories.
“ML1 was worn towards the crown of the head. So you really had to provide instruction. It was kind of like, ‘Wear it a couple of fingers above the ear,'” Temple says. He set out to make sure the second-generation device had a more familiar fit, as simple as putting on a pair of eyeglasses. “People know how to wear glasses. Your ability to put on this new device, which is a new relationship with a piece of technology on your head, should be incredibly intuitive,” he says.
To remain stable, though, it couldn’t just have the two folding arms of a typical pair of glasses. The device has to wrap all the way around the skull, which required the creation of a novel telescoping armature that spreads open for placing over the head, pivoting at just the right angle to stay snug on different head shapes and sizes.
And as the designers quickly found, there are many, many different head shapes and sizes.
“We scanned a lot of heads”
Across ages, sexes, ethnicities, and hairstyles, people’s heads have remarkably varied dimensions. “All these factors have very tangible consequences on the ability to wear the device,” says Danny Carson, head of user/design research for Magic Leap.
To accommodate the global diversity of heads with a single device—a simpler and more profitable way to manufacture and market a product—Carson’s team dove deep into cranial physiology. “We scanned a lot of heads,” he says.
Using a submillimeter-accurate scanning device on nearly 2,000 heads across demographics, the team gathered millions of data points on 18 different facial landmarks like the locations of the eyes, corneal apexes, nose bridges, temples, and ears, as well as 19 individual hand-measured metrics around the head related to the skull.
“What we were able to do is map out all the differences between these points that we see from a statistical standpoint and design in a way that would accommodate the most significant drivers of fit,” Carson says.
That detailed analysis gave the design team the rough guidelines it needed to shape the form factor of the device. Next they had to figure out how to cram that shell with computing and optical technology that’s powerful enough to assist in a heart surgery.
Prototyping and prototyping and prototyping
“This used to be one of the most secretive rooms in the headquarters,” says Temple on a recent video call, as the camera scans around a room full of the building blocks of the Magic Leap 2. There are dozens of prototypes of the headset, compute pack, and handset, each an increasingly refined shot at packing in that balance of wearability and functionality. The earliest prototypes date back to 2019, just a year after the release of Magic Leap 1.
The handset prototypes go from a blocky joystick to a slightly angled trigger intended to follow the natural posture of the wrist. The compute pack’s edges smooth out into a barely domed circle with a pocket-ready slit and metal-clipped shoulder strap. The cord, chunky and stiff in the Magic Leap 1, is now a fluid fiber-optic cable.
Magic Leap tested each version of the headset’s wearability on a variety of users for time periods ranging from five minutes to three hours. Carson says the design and user experience teams learned a lot of literal pain points from those longer tests, identifying problem areas that revealed themselves only upon extended use.
“Early on we felt some pressure on the side of the head where some of the architecture was near a speaker,” he says. “We actually had to, from a mechanical engineering perspective, go in and make some changes to reduce that.”
For Magic Leap’s enterprise customers, long-duration wear is likely. When a heart surgery can last six hours or a factory-floor training session can take an entire day, tiny improvements can make the experience of wearing the headset for an extended period tolerable.
In 3D-printed examples that simulated the weight of the necessary hardware with tiny ball bearings packed inside, the device’s evolution was another game of millimeters, with nose bridge and forehead pads added to balance the weight and adjust for the varied shapes of the face and the head. The designers worked over the course of years to position the lenses perfectly in front of the eyes, to tune the sensors to capture the physical world through those lenses, and to overlay a finely focused display that floats in space without occluding reality. Two tiny cameras are pointed back at each pupil to ensure what’s shown on the lenses stays in focus as the eyeball moves about.
The spacing went down to eyelash level. “We tried hard to bring the optics as close as possible to the eye, which enables us to have a much bigger field of view,” says Gary Natsume, Magic Leap’s SVP of design. “That makes it more comfortable to wear, because the center of the mask comes closer to your face.”
An expansive view is helpful for a doctor who is trying to sew up a patient or an emergency response crew trying to prioritize rescue efforts, but it’s not totally necessary. For these specific use cases, Magic Leap 2 doesn’t have to be fully immersive to be effective. Eventually, though, customers will want a headset that frees them from the boxes of content they are accustomed to with TV screens and phones.
For augmented reality to really catch on, a bigger field of view will be crucial. A seamless digital overlay on the real world is the core value proposition of AR, and when the edge of that virtual world is easily visible, the magic is sort of lost.
Still, the field of view is significantly expanded in Magic Leap 2 compared to the first-generation device; it now spans 70 degrees from left to right, compared with about 50 degrees in version one. The view is also much bigger vertically, which Natsume says is the more important improvement. “People’s necks move right to left much easier than up and down,” he says. Reducing the movement required to perform a task, be it repetitive or potentially life-saving, was a priority.
“The last thing we want is people looking like a robot”
The view through the lenses was a prime concern, but the designers were also focused on how the device looked from the outside. Perhaps sensitive to the “glasshole” pompous nerd problem associated with Google’s early venture into smart glasses, Natsume says the new version of the Magic Leap headset was designed to evoke normal eyewear, using familiar elliptical shapes and lens surfaces with a visible coating to cut down the general weirdness of seeing someone wearing a computer on their face.
“The subtle reflection is very important for us to make sure that people read the profile of the glass. There are a lot of technologies hidden behind it, and the last thing we want is people looking like a robot,” he says. “We want people to perceive them almost like a pair of glasses, that it’s something familiar, not alienating. . . . It’s a very exotic product but we want people to feel comfortable in a public situation.”
The design of Magic Leap 2 is a notable step up from the first version, which had bulging circular lenses like a pair of steampunk Oakleys. The lenses in version two are much more familiar, if still bogged down by the subtle yet still conspicuous sensors that surround them.
For now, the company hopes this performance will meet the needs of three main industries: healthcare, manufacturing, and the public sector. Example applications during the recent demonstration included a drone’s-eye wildfire visualization for real-time firefighting, a fleet-monitoring tool for taxis and delivery vehicles in San Francisco, and a clever tool for dimming the natural light in a space like an operating room and brightening the digital overlay of something like an MRI.
The scrim the device places over the real world can either be a small reference point in a user’s periphery or an all-encompassing digital reality. And though this version of the device was designed with enterprise users in mind, it’s now on the open market and in the hands of developers to build whatever they see fit, be it a game, a public service, or a tool of national defense.
Time will reveal how useful this augmented reality device is, if the metaverse is a place people actually do want to go, and just how bearable it is to wear a half pound computer on one’s face for hours at a time. Whatever happens, the design process hasn’t stopped. Magic Leap’s designers are already at work on the next iteration, one that will likely be lighter, more powerful, with a wider view, and maybe even more comfortable.
“This is a technology category that’s really in evolution,” Natsume says. “Every day we’re working through how to make this better.”
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.