Last October, Fast Company broke news about a stealth startup named Magic Leap that had raised $542 million in a round of financing led by Google. Names attached ranged from the mobile hardware makers at Qualcomm to the special effects studio Weta.
But what the heck were they building? Augmented reality (AR) glasses were our best guess.
Now, Magic Leap has begun filing patents, which give us a much better look at their plans. Glasses? Yes. Glasses that can map their surroundings in a massive cloud database shared by the world’s Magic Leap users. Glasses that project uncanny images right into a wearer’s line of sight. Glasses that can recognize our gestures. Glasses that will make the entire world a canvas for a new digital interface that follows us everywhere we go.
Their latest application is a 180-page opus for user interface, filled with a surplus of imaginative sketches illustrating the platform’s potential. Will all of the ideas end up in some Magic Leap shipped product? Of course not. But the application still gives us the best peek into what Magic Leap is dreaming up for us next. So we pulled out 48 images from the legal documents, and will do our best to walk you through them in good old plainspeak here.
The hardware appears to be a pair of glasses with fiber optic projectors. Images are painted directly in front of your eye.
Hand gestures within your line of sight can pull up menus and issue commands.
Here we see the first of many “totems” in the documentation. This is a bracelet where each charm becomes a button to load social media.
Totems can be dumb, non-descript objects in the real world, but painted with a digital skin in Magic Leap’s augmented world. Here we see a series of digital “keys” that could unlock mechanical locks or digital ones floating in space.
Totems can also take the form of controllers. Again, they can be built without electronics–the Magic Leap would read the gestures you make with them instead.
Here, a petal totem–the petals can actually be AR constructions–each with their own button.
A haptic glove–which provides vibration feedback–could be used to give the sensation of touching physical objects in midair.
But many totems will be objects we already know–like this paintbrush, which is basically a stick that allows you to paint in digital ink.
A physical keyboard wouldn’t need to have any specific keys at all. Magic Leap could remap a keyboard on the fly.
With virtual interfaces, a screen is always at your fingertips.
One such manifestation is a radial dial that you see on the floor.
Another concept the application comes back to again and again is this waist-high hula hoop.
With the Magic Leap’s glasses and mapping technology, you could place virtual objects within real spaces.
You could also place screens anywhere.
Pinch-to-zoom? Nope. Now it’s pinch-to-expand the entire screen.
It’s a gesture the application features again and again.
Here we see how easy it would be to hang a screen on the wall.
But the illusions aren’t limited to your environment. Your hand could become a controller all its own.
Remember the iPod scrollwheel? Magic Leap imagines the design plastered on your hand to cycle through apps.
The office as you know it would no longer require a physical computer.
What’s going on in this wild scene? It’s actually a demonstration that the Magic Leap could store 2-D icons and images, or it could store 3-D icons and images. You could explore your files and media in 3-D, grouping files and contacts as a mini universe.
Here we see a virtual desktop that sits on top of a real desktop. The Magic Leap can call up these special menus contextually, meaning one interface could live at your desk, and an entirely different interface could live at your couch.
Let’s call this what it is: interface porn.*
In his living room, this man has pulled up the “pod” interface–what the application calls a “mini work station” for the user that ensconces their entire view.
In that same context, Magic Leap can reimagine the space for working on a model–the 3-D version is on the table, the 2-D version is a wall projection. And various related files float on the walls.
I was going to look up the annotation on this image, but I’d rather pretend the guy is playing dolls while watching football.
Again, we see that hula hoop interface make a comeback, even on the couch.
Notice that watching Netflix in a media room can be a totally different experience from ESPN. Blank walls can be painted with virtual posters, contextualized by the app.
But what about socialization? Never do yoga alone at home again when virtual people can appear beside you.
The potential for exercise is tied deep into the gamification themes within the Magic Leap application. There are a lot of mundane tasks that become a competition for a high score.
That lawn mowing gig got you down? Magic Leap will allow you to butcher virtual gophers instead.
Chopping veggies? Not without a bonus multiplier you aren’t!
This bit of interface mixed with gamification is really a strange one. Can you guess what you’re looking at? That’s actually supposed to be the HUD for a firefighter’s glove.
So what about shopping? Say you want to buy a book but don’t know what sort of reviews it has. The Magic Leap could scan the covers and float reviews right over top.
At the grocery store, the cart’s handle becomes its own interface.
But what about the kids? They can play a game, hunting for a dude named Gerald.
Gerald was hiding in the produce all along! That’s so Gerald!
Any object can become a portal to more information–and its own form of advertisement.
This is a “friendly monster” bursting through cereal boxes. It could prompt the child to initiate a game–a game that maybe the adult doesn’t even see.
Meanwhile, Mario Batali can appear anywhere!
Moments like this one, where the context of a wine case prompts a wine buying guide, may not look as magical as some of Magic Leap’s other ideas, but they’re amongst the most useful utilities in the application.
And no matter how bad Magic Leap operates in the real world, checking out can’t be worse than the automated checkout lanes at most grocery stores.
But in a dystopian twist, notice that the little girl doesn’t see her mom checking out. Why? She’s still playing Find Gerald, disconnected from her mom’s experience.
Finally, we make our way to the hospital, where a woman is about to have heart surgery. Here we see one of the few social use cases demonstrated in the patent application. Everyone around the table is able to see the same image, centered on the same coordinates in the room, rendered in each person’s glasses individually.
Magic Leap could guide doctors through surgery.
And when it’s over, it can visualize how the surgery went . . .
. . . or it can just tune out the bad news.