Carl Ledbetter simply can’t help himself. The cheery director of design at Microsoft, who leads the design of the company’s Xbox game consoles and HoloLens mixed reality headsets, is in a particularly good mood today. With a belly full of pumpkin pancakes on a crisp fall day and a grande Starbucks at the ready, he’s sharing, for the first time, how the company built its newest product: the HoloLens 2.
He’s also sharing, at last, that it’s actually shipping. Announced in early 2019, nearly a year ago, the first HoloLens 2 orders will be delivered today for $3,500.
He’s not supposed to be telling me everything, not yet. I still have four more interviews set up with members of the HoloLens team, and big companies like Microsoft carefully plan these narratives for members of the media such as myself, sharing a drip of details that culminates to a grand reveal.
I’m captivated, because I don’t know that I’ve ever seen such a seamless combination of hard and soft parts in a piece of electronic hardware, let alone the dozen different materials I can quickly make out with my eye.
None of these individual details matter on their own, of course. It’s the experience that they all add up to. The HoloLens 2 weighs just 13 grams, or about half an ounce, lighter than the HoloLens 1. But it’s measurably three times more comfortable to wear.
How? That’s the twist.
Building “instinctual interaction”
The first HoloLens launched in 2016. For Microsoft, it was a top secret project that, somehow, actually stayed secret until launch. The device was Microsoft’s take on augmented reality, floating holograms in the real world. It was, on one hand, amazing. No one had ever shipped such a device before. On the other, it was awkward in just about every other way.
The UI was confusing. You couldn’t grab or poke the holograms as you’d expect. Instead, you had to awkwardly hold a finger vertically and air tap, a gesture I might best describe as something you’d do playing with a child, using your index finger to speak as King Friday. The field of view was constantly disappointing. Holograms were consistently cropped in your view, destroying the illusion. HoloLens 2 addresses these shortcomings with twice the viewable area and better onboard AI, which allows you to grab corners of a hologram to stretch it out—or just snatch the whole thing with your hand.
Another highlight of the new model? Some truly ingenious on-boarding UX. When I first put on the device, hold out my hand, feeling, obviously, a bit silly reaching for air. And that’s when a fluttering hummingbird streaks into the scene. It darts around before landing on me. The hummingbird is the first thing HoloLens 2 users will experience when putting on the headset out of the box, and one of those magical moments you hear about in mixed reality. But while I’m giggling at the bird, the system is silently calibrating to recognize my hands.
Microsoft dubs this approach “instinctual interaction” or “instinctual design,” and it wants to build it into every part of the HoloLens experience. The idea is to make learning the UX feel natural, rather than instructive, right down to how you put on the new headset (it’s now like donning a baseball cap, thanks to a painstakingly iterated industrial design).
Truthfully, the hardware itself is still the biggest single barrier to the mass realization of mixed reality. The first HoloLens was a self-contained headset computer! Amazing! But, wear it for long, and you realized it was front-loaded. It would begin to tip down onto your nose, forcing you to readjust or look through it like a pair of ill-fitting reading glasses.
Microsoft heard all this feedback from its early adopters, and acknowledged that in addition to housing more sensors, more computational power, more battery, and a bigger display, the HoloLens 2 design also had to be more comfortable.
“I was like, let me make sure I got this right,” recalls Ledbetter with a laugh. “A lot more stuff, on your head.”
“Technology should not be fashion”
Ledbetter, along with Microsoft technical fellow Alex Kipman, the mastermind behind Microsoft Kinect who you could also call the father of HoloLens, both agree that the ultimate incarnation of HoloLens would be a pair of glasses that was so thin and light it would be indistinguishable from what you’d buy at LensCrafters. “But the reality is, the technology just isn’t there yet,” Ledbetter says.
Kipman doesn’t want to pretend it is, either. That’s why HoloLens 2 doesn’t solve the problem of weight by making you wear a little computer on your belt, like the competing Magic Leap. And it doesn’t give up on the promise of holograms with a smaller, less immersive device, like Google Glass did. The inventor takes a philosophical approach to the headset, arguing that you have to embrace what it is as a tool, not what it could be as an accessory.
“[HoloLens is] not made to be fashionable,” Kipman says. “Technology should not be fashion. Fashion is ephemeral and expires. Hardware should be timeless . . . if aliens come to Earth thousands of years from now, do archeology, and find these devices, they should understand . . . that it’s something meaningful.”
So for HoloLens 2, Microsoft began its ergonomic research anew. It 3D-scanned 600 individual heads of people with varying age, gender, and race, in an attempt to come up with the thresholds of inions (the protrusions on the back of your head) and foreheads, both of which can vary wildly by person. All of this work informed prototypes for new fit systems, because HoloLens couldn’t feasibly be custom-made for different sized heads, like shoes are for feet.
I look at some of those prototypes, sitting on the table in front of me, with Ledbetter. They are lovingly hacked-together devices, with exposed skeletons and foam. They’re so raw that they’re unintentionally stylish. One model has springs, and a complicated ratchet adjustment on its temples to tighten down. Another has webbing with big holes and coral foam—I point out that it looks like a HoloLens by Yeezy.
A computer shaped by anatomy
Since the headset was being designed before the actual HoloLens 2 laser display was finished, Ledbetter’s team had to be creative in testing the project with low-fi proofs. They would 3D-print long tubes and attach them to the head rigs, to simulate having a tiny window lined up perfectly in your eye. In another approach, the lenses were mocked up in acrylic. Wearing these headsets myself, they pinch at my hair and slice into my skin. But in terms of simulating the optics of the real device, they work as intended.
One other major design issue these prototypes were constructed to test was weight. What the team had realized all too well was that the HoloLens 1 was front heavy. But by studying cultures around the world, known for carrying heavy loads in baskets upon their heads, they realized that fixing HoloLens might not require cutting weight. What if they just balanced it better?
In an extensive prototyping process, they weighed down these various HoloLens contraptions with dumb, dead mass, just to understand what the comfort gains and losses would be with varying designs. Human factors specialists on the team ran electrodes to the back of testers’ necks to test how their bodies responded. What they found was that if HoloLens was balanced about 50-50 to the front and back of the head, they could reduce the activation of supportive neck muscles by three.
But from an industrial design perspective, doing that was easier said than done. It required splitting the headset’s single computer vision system into two sections, one in the front, one in the back. To connect them, wiring had to be run through the headband of the device—a span of just a few inches—as opposed to soldering all of the components onto a single circuit board, like you would when designing a phone. Any extra wiring was a serious UX concern, because, as Kipman puts it, in mixed reality, where a device is reading and responding to the movement of your eyes, head, and body at the same time, “the speed of light is too slow.”
And then there were the final concerns: How do you get the HoloLens on? How do you take it off? What the team handed on was a design that, after being adjusted one time, could be taken on and off just like a baseball cap—possible through a combination of ergonomics, forgivingly stretchable materials, pivoting components, and that aforementioned back adjustment tool. And as for the laser lenses themselves, those can moved aside, too.
Ledbetter looks at me eagerly as I wear the device.
“Just tilt up the front,” he urges me, miming the motion of sliding sunglasses up on my head above my eyes. So I tentatively push the glasses up. They seem like solid plastic. It seems like I’m about to break a priceless preproduction laser display. And yet, the lenses flip right up like a welding cap, leaving the rest of the HoloLens 2 firmly anchored to my head.
“Oh. I don’t know,” I say, suddenly feeling even more self-conscious. “I’m not sure about this one.”
“I know,” says Ledbetter.
“I look cool? Is that what you’re saying?” I offer sarcastically.
“No, not at all! This was the debate!” he laughs back. “Someone mocked up this thing, and as designers we’re like, no, no, that’s really geeky and kind of dumb. But what we found is, this is functionality. Function first, practically first. And what we found out is that flip-up feature, as funny as initially it seems, becomes ingrained in behavior.”
Perhaps it does. Though to my surprise, I personally don’t mind talking to people while they or I am wearing the device. I believe that’s because the best usability decision Ledbetter’s team made was to shave off some of the extra plastic that would have blocked your peripheral view. This creates an opening and naturalism to conversation otherwise lost behind bulkier headsets.
In any case, I’ve only worn the HoloLens 2 for about 20 minutes during our chat. Unlike the first model, it’s comfortable enough that you really can forget you’re wearing it. Or at the very least, it’s not always distracting you with its weight and fit.
No, I don’t look cool wearing the HoloLens 2, but to be fair, Drake wouldn’t look cool wearing the HoloLens 2, either. And he’s really not supposed to.
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.
