After mentioning its interest in augmented reality (AR) publicly on several occasions, Apple still managed to surprise many of us on Monday by announcing a new tool–ARKit–that will let developers create augmented reality experiences for the iPhone.
This is a perfectly Apple way of entering a new tech space. Yes, plenty of big Apple platforms have launched the hardware, then later let developers create on it–iPhone, Apple Watch, Siri, tvOS, for example. But Apple is known for being patient and cautious about entering new tech areas, and leading with a development kit is just another way of doing that.
Apple says the AR experiences that developers create will run on any iPhone with an A9 or later chip. The ARKit AR, in fact, is already starting to show up on Twitter. And, admittedly, they look impressive.
— codyb (@CodyBrown) June 7, 2017
“This is shot on a 6s,” says developer Cody Brown in a reply to the tweet. “Can only imagine what they will do with a better depth sensor.”
Yep. I’m betting Apple’s real intent is to let developers start getting their feet wet developing AR for future iPhones that create better AR using more and better sensors.
Developers, ultimately, are not going to be satisfied with the limitations imposed by the basic smartphone camera of most existing iPhones, which were built for taking photographs, not measuring 3D space (with the possible exception of the dual-camera iPhone 7 Plus).
That’s why it makes sense that at least one of the new iPhones announced this fall will include a 3D sensor on the back to more accurately capture the fine layers of things situated closer or farther away from the camera. Only then can it place digital imagery within that environment in a convincing way.
A key question is whether ARKit be used to create AR experiences that exceed the limitations of existing iPhones. Can ARKit create experiences that would look better with the help of an additional 3D sensor on the device they’re running on? Since developers are just getting their hands on the new SDK, it’s hard to know. But it would be strange for Apple to release a development kit that tied the quality of its product to the limitations of the existing consumption devices. Apple, if you haven’t noticed, isn’t shy about moving on to new and better technology standards, even at the expense of people who own older-generation devices.
Anyway, here's a chameleon trying to catch bubbles. pic.twitter.com/RSSKV1nFYY
— Paul Bronks (@BoringEnormous) June 7, 2017
The new 3D sensor coming in the iPhone 8 (or “iPhone X” 10th anniversary edition) will be supplied, at least in part, by Lumentum. A recent channel check indicated that Lumentum will be supplying about half of the 3D sensors, while II-VI and possibly one other vendor will be proving the remainder.
Until the announcement of ARKit on Monday, it was unclear whether the new 3D sensor would be located on the front of the phone for use in facial recognition (for unlocking the phone) or on the back of the phone for AR. Now it seems more clear that the sensor will be on the back of the phone.
Releasing the SDK now will give developers ample time to create cool AR apps before the appearance of those new phones this fall. It’ll also allow a little more time for the general public (those who haven’t played Pokémon Go, anyway) to understand and warm up to the concept of AR.
Of course Apple will develop some of its own native AR experiences, which will likely begin showing up in iOS 12. In Maps, for example, AR may be used to put digital place markers and other information within a sort of 3D “Street View” environment.
Google has been working on AR for years with its Tango software. Apple is coming at AR from a very different angle, however. While Google has, so far, depended on partners like Lenovo to create phones with the multiple sensors needed to support AR, Apple brings its own consumption device with the iPhone. Because there are already hundreds of millions of iPhones in the wild that could run the ARKit apps, Apple says ARKit is already “the biggest AR platform in the world.”
But claiming the biggest AR platform comes at the cost of lowering the threshold for admission, at least where the devices are concerned. ARKit experiences running on single-camera phones probably won’t be as good as those running on multiple-sensor phones (like Tango) or ones running on powerful dedicated devices (like Microsoft’s HoloLens and, potentially, Magic Leap).
Apple, of course, won’t be comfortable with that competitive dynamic. It will want to build future iPhones for superior AR. And unless Apple has built software so magical that it can create great AR on existing single-camera iPhones, it’ll do that by adding new 3D sensors in the iPhone 8 and beyond.