What's So Hard About Augmented Reality, Anyway?

Yesterday I posted about Nokia's anemic vision of augmented reality. And while we should all hold the world's largest phone-maker to a high standard when it comes to technology like AR, it's worth discussing why the future of these apps is so difficult to solidify.

Sure, our smartphones have GPS, and cameras, and Internet access. And it sounds like the iPhone will expand its dev-friendly API to include AR framework in version 3.1. But useful AR is much more than all that.

For one thing, AR is more than just overlaying information on a view of the world. True augmented reality knows what it's seeing; it can read the visual language coming into the camera to know that a building is a building; a person is a person. Using GPS and a compass to slap some labels on Google Street View doesn't cut it. (Not that apps like Wikitude, below, aren't useful.)

Anaylzing a video stream is not only tough to program for, but also very processor—and memory—intensive. A phone like the iPhone 3G, which has a relatively puny 128MB of memory, can only devote about 30 to 60 MB of RAM to any given app before the OS kills the instance for being too greedy. That means that AR apps must be built with aggressive memory management in mind, something that's tough on mobile phones. As good as Apple's IDE is for designing for iPhone, you still can't use the garbage collection build setting, which makes memory management for complex apps much easier. (If you are wondering what the hell I'm talking about, check out this series of accessible videos on iPhone memory, below.)

Another roadblock is how to handle temporal updates issues. If users of an app are supposed to be experiencing the same dynamic layer of information, then the cloud that manages these changes will suffer a barrage of constant micro-changes: tweets, photos, locations, tags, Wikipedia alterations and so on. That's a hell of a lot of information for your phone to be downloading every time you launch an app. It's also a lot to be constantly uploading.

Also, what about openness? As this handy RWW article points out, the only truly useful AR apps will have enough interoperability to be able to include Android, iPhone, Blackberry, Palm and computer users all together, no matter which AR client they happen to be using. That means either one provider monopolizes and builds a great API (ala Twitter) or we're stuck with several behemoths with all their info in a silo (think MySpace versus Facebook.)

Then there's the question of how to integrate these apps into society. How do we explain AR to our non-geek friends? How do you get your girlfriend or boyfriend using it? Video's like Layar Japan's don't really help. Not to mention: how do you walk around with your phone right in front of your face all day?

Add New Comment

1 Comments

  • Matt John

    It's the mobile phone market that will drive Augmented Reality as it has both the ease and use via the mobile phone and the revenue from advertisers looking to focus on certain demographics.

    Matt John
    --------------------
    dealer of baseball caps!