You wouldn't immediately suspect that Yelp's iPhone app might be a gift bestowed upon us by a benevolent superhero from the future. Load it up and the program's in its Clark Kent garb — a useful-enough guide to local restaurants, bars, and merchants.
Then you notice a button labeled monocle in the right-hand corner. Hit it and the screen displays a live feed from the phone's camera, showing exactly what's in front of you — with one big difference. Aim the camera at a local storefront and Yelp superimposes a star rating on the image. Use Monocle in a hot neighborhood, for instance, and point it at every restaurant for a quick appraisal of the best food in the area.
Yelp's app is one of the first "augmented reality," or AR, programs to debut on the iPhone, and though it can be handy, it's most useful as a sign of what's to come. Throughout the summer, YouTube was the place to see a vision of that future, as programmers from San Francisco to Malmö, Sweden, uploaded demonstration videos depicting such feats as recognizing a face at your high-school reunion while his social-networking pages pop up, or traveling back in time to view the Colosseum as it once existed.
And then there are the really forward-thinking ideas. Babak Parviz, a bio-nanotechnologist at the University of Washington, has been working on augmented-reality contact lenses that would layer computer graphics on everything around us — in other words, we'd have Terminator eyes. "We have a vast amount of data on the Web, but today we see it on a flat screen," says Michael Zöllner, an augmented-reality researcher at Germany's Fraunhofer Institute for Computer Graphics Research. "It's only a small step to see all of it superimposed on our lives." Much of this sounds like a comic-book version of technology, and indeed, all of this buzz led the research firm Gartner to put AR on its "hype cycle" for emerging technologies — well on its way to the "peak of inflated expectations."
Marketers are as much to blame as geeks for the overheated environment. To date, the most prominent AR practitioners have been ad agencies, and Best Buy, GE, and Procter & Gamble have run campaigns. Most of them, unsurprisingly, were more gimmicky than useful.
The central questions: Will people take to this mode of navigating information in the same way they've embraced social networking? Or will augmented reality suffer the fate of online virtual worlds such as Second Life, which attracted a torrent of attention but proved too cumbersome? Whether augmented reality emerges through this hype cycle depends on both technologists and marketers to peel away the great expectations and find something real.
Augmented reality isn't new. The technology has been used for years in military projects as well as public spectacles such as museum exhibits and trade-show booth demos. The yellow first-down line superimposed on televised football games is an example of augmented reality. So why all the chatter now? "There have been a couple of game-changing events," says Greg Davis, North American general manager of Total Immersion, a decade-old French company that built these first-generation AR installations. "Consumers have access to AR on their home PCs, and now on their mobile phones as well."
At home, consumers can augment reality through a Webcam. Hold a printed barcodelike "target" up to the camera, and the screen, rather than showing the piece of paper, will meld the video input with 3-D graphics.
Webcams, of course, are tethered to your computer, which is why mobile devices have provoked so much interest in augmented reality. Modern smartphones can determine their location through GPS and an internal compass, they can download data through mobile broadband connections, and they have powerful graphics-processing capabilities — all the ingredients for rudimentary AR.
In September, Apple updated the iPhone's operating system, allowing third-party apps to superimpose graphics on live video. Although Apple still limits how far developers can go with AR — in particular, the phone's programming system prohibits apps from performing sophisticated image analysis on input from the camera — the new OS is expected to usher in a rush of augmented-reality apps.
As a kind of prelude to the wonders that may be available on the iPhone, many AR firms have built versions of their apps for Google's Android platform. Among them is Layar, an Amsterdam-based startup whose Reality Browser aims to become the Netscape of AR — the program that defines for the masses the everyday wonders of augmented reality. Layar's app, which was downloaded more than 50,000 times in its first week in the Android store, uses the Web as its operating metaphor: Load up the browser, and then pick third-party "layers" to superimpose on top, the same way you'd visit a page in Firefox.
Layar has built an open-ended platform that lets developers quickly add content. There are now dozens of layers, including ones that access Wikipedia, Flickr, and Twitter. Maarten Lens-FitzGerald, Layar's cofounder, is particularly taken with the tourism apps, such as a World War I layer in the north of France that lets people see pictures and info about battlefields as they're standing on them. "You're at an intersection; hold up your phone and see a picture of that intersection from 100 years ago," he says. "There's a temporal difference between reality and the picture you couldn't see before."
"Augmented reality is going to be huge. It might even be as big as the Web." — Layar cofounder Maarten Lens-FitzGerald
Still, today's phones are limited in their AR powers. "We were trying to superimpose an animated dragon the other day," Lens-FitzGerald says, "and we found you really can't do that." But as the technology improves, AR apps will be able to recognize faces and physical objects and render detailed 3-D animation sequences. "And that's when we'll see levels of engagement rise," Lens-FitzGerald says. "People will play AR games. Instead of seeing the big apartment block in front of you, you'll see a castle."
At the moment, AR's commercial potential is not in apps or layers, but in consumer goods and advertising. Total Immersion has teamed up with Mattel to produce what they call the world's first AR toys — action figures based on the forthcoming James Cameron movie, Avatar. The figures come with an "i-Tag" that allows users to see 3-D versions of their toys playing in their own rooms. Total Immersion has also produced AR baseball cards for Topps, and its technology can be found in a number of big DVD releases, including Transformers: Revenge of the Fallen. In each case, marketers know that consumers won't buy the products just for the AR but hope that the gee-whiz appeal influences the buying decision.
The ad industry has adopted augmented reality for the same reason. Procter & Gamble recently ran AR-powered magazine ads for its Always brand of feminine-hygiene products. When women interacted with the ad, they saw an animated rabbit jump out of a hat — in theory, bolstering the brand's tagline, "Works like magic."
The danger is that the magic will wear off rather quickly. An AR ad looks pretty cool the first time, but once every brand wants you to hold something up to your Webcam... .
"It's the flavor of the day," says Jonathan Sackett, the Martin Agency's chief digital officer. "I remember when every client had to have a Facebook app, then a viral ad, then it was an iPhone app, and now it's augmented reality."
The Martin Agency isn't immune either, but at least its AR campaign for Wal-Mart's Your Zone line of furniture for teenagers was useful. Kids could move around the AR target to simulate a bed, dresser, chair, or other Your Zone item, seeing the piece just as it would look in their room. "For us, it was very practical," says Sissy Estes, a VP and associate creative director at the agency, "to show the customer that there are so many different combinations available."
Wal-Mart declined to disclose engagement figures on the campaign, and even if it had, usage metrics — how people respond, and whether these systems can keep people entertained or communicate a branding message — are difficult to decipher. In August, Best Buy included an AR marker in its Sunday newspaper circular that reached 43 million people and only about 6,500 tried it out at BestBuy.com (the company said it's happy with the response). Last summer, a Total Immersion campaign for Transformers 2 got 80,000 visitors to spend an average of three minutes interacting with the AR app. Are these returns good or bad? "There aren't any benchmarks," admits Total Immersion's Davis.
The AR industry is wary of converting every imaginable Web thing into an augmented-reality experience. "But on the route toward the best AR for all, that's part of it," Layar's Lens-FitzGerald says. "We need to run and fall and get up again and get going." In other words, get ready for augmented reality everywhere. And after that, we'll see AR where it makes sense.
A version of this article appeared in the November 2009 issue of Fast Company magazine.