Apple last month launched what could soon end up being the largest augmented reality (AR) platform in the world, and its rivals may have a tough time answering in kind. Apple, it turns out, is probably the only company that could launch an AR platform with the kind of performance and broad consumer reach of its offering, ARKit. That’s because it controls so many parts of the software and hardware involved–just like Jobs envisioned.
While Google has spent years working on its own AR platform, Tango, having it opened it up to developers back in 2014, Apple now seems poised to swoop in and carry AR into the mainstream, leaving the kids from Mountain View in the dust.
The AR apps developers create for ARKit will immediately work on many millions of iPhones out in the wild. No special 3D sensors required, just the iPhone’s camera, and an A9 or A10 chip inside.
Tango AR, meanwhile, works only on a couple of phones–Lenovo’s Phab 2 Pro and Asus’s ZenFone AR. Tango may be more accurate at mapping objects and rooms because it requires the host device to have active 3D sensors to measure the precise locations of surfaces and objects within the frame. The device also has to have a large battery to power the 3D lasers.
“Google must realize they dramatically overshot with Tango,” says IDC analyst Tom Mainelli, referring to the platform’s rigorous hardware requirements.
ARKit, on the other hand, relies on no special 3D sensors on the back of the iPhone. ARKit apps can place digital objects in mid-air, and on horizontal surfaces. What ARKit can’t yet do is attach content to real-world objects in the frame. Nor can it make digital content interact with more complex surfaces (it can’t make a virtual car race across the floor and bounce off a piece of furniture, for example).
Despite those limitations, the quality of the AR we’ve seen developed on ARKit is surprisingly good.
Why ARKit Works
The secret sauce isn’t much of a secret. ARKit experiences look compelling because they are supported by several key components of the phone–the camera, the processors, and the phone’s various inertia sensors (accelerometer, gyroscope, etc.)–all working in tight concert together. They’re tuned and optimized to precisely determine the position of the iPhone relative to the environment, as the user moves it around. Apple can can do this because all the components are made by the same company, Apple.
Apple can bring a lot of processing power to bear, too. ARKit processing happens on the A9 or A10 system-on-a-chip (SOC) within the iPhone. The dual-core application processor on the SOC works with both the M9 motion processing chip (which processes data coming in from the motion sensors) and with an image signal processing chip (which processes image data coming in from the camera). This all happens extremely quickly and in a way that doesn’t rapidly drain the battery or overheat the phone, explains Jay Wright, president of PTC’s Vuforia AR platform.
Apple says millions of computer vision computations are already happening during ARKit experiences on the iPhone. Right now, however, “computer vision” software can only understand the rough outlines of objects in AR settings. “This is because it is not yet capable of determining an accurate 3D position or the accurate contours of an object,” Wright explains. “Both are required for the more interesting AR use cases.”
Right now the image signal processor (ISP) does much of the computer vision processing needed to recognize imagery seen through the camera lens. Apple is said to be working on a dedicated AI processor, which might take over the bulk of the image recognition duties for AR. Other reports have said Apple is working on its own graphics processing unit (GPU), the kind of chip most commonly used for heavy-duty computation and AI algorithms.
https://www.youtube.com/watch?v=KxTLOXzhqzA
A Sea Of Androids
Google can’t orchestrate the under-the-hood workings as Apple can because it enjoys far less control over Android devices. Google, naturally, decides what components go into its own Nexus and Pixel phones, but for the most part, Android phone makers like Samsung decide what chips, cameras, and sensors to put in their own phones.
Google may have been able to optimize the Tango experience to the Lenovo and Asus phones because it worked closely with the OEM to choose and optimize the components. Typically, however, Google can’t reasonably tell phone OEMs what camera would work best for its AR platform, because consumers use the camera mainly for photos and video, and the phone maker naturally wants to put the best camera it can afford into the phone. It’s a point of differentiation, and something to brag about in marketing materials.
So there are hundreds of different Android phones out there, all with different innards. The combinations of components are endless. So much for carefully optimizing a set of specific components to work together. Add to that the fact that those phones run a number of different versions of Android.
As a result, there’s no practical way to design an AR platform that can optimize the components in any one of those phones to create a comparable AR experience.
What Can Google Do?
To defend against the Apple tide, Google might start by building an AR software platform–a sort of “Tango Lite”– that doesn’t require the host device to have any special 3D sensors or processors. The platform would work to calibrate whatever sensors, cameras, and chips it found on the host device to create the best AR experience possible. It might do better on some devices than others, depending on sensors and processing power. That’s not so different from what Vuforia and Wikitude have been offering AR developers for a long time via their software development kits (SDKs), and it could also help to further democratize AR beyond Apple’s walled garden.
Google could also buy. “One way they [Google] could be competitive with ARKit sooner rather than later would be to throw support behind—or buy and integrate directly into Android—an existing AR platform,” Mainelli says, pointing to Vuforia and Wikitude as two possible candidates.
Vuforia–currently the largest AR development platform (by number of apps)–was born and raised inside Qualcomm before being sold off to PTC. The platform lets developers create AR apps for iOS, Android, and the Universal Windows Platform. Wikitude allows developers to make apps for iOS, Android, and JavaScript.
Vuforia has worked with Google to make its platform compatible with Tango, but Wright says he hasn’t talked to Google about the possibility of an acquisition. Nor do we know of any such talks going on between Google and Wikitude.
Whether it’s one of those options, or something completely different, don’t be surprised to see Google make some sort of strategic move in the direction of widening the appeal of its own flavor of AR.
The way things look right now, Apple’s ARKit may end up being the second big inflection point for consumer augmented reality. The first was last summer’s obsession with Pokémon Go (a project that grew out of a Google startup, Niantic). If Apple manages to bring another wave of attention back to AR, and if AR becomes a bigger, more sustainable part of the smartphone experience, Google might find itself trailing Apple in a way that becomes increasingly painful.
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.
