Fast Company

Nokia Introduces Augmented Reality App for Movies

Augmented reality has been slowly moving beyond the realm of science fiction into real life, but it just got a large boost from Nokia. The Finish cellphone maker has introduced a public version of an app dubbed "Point and Find" for its legion of cellphones, and it automatically recognizes real-world objects for you.

Point and Find works as you'd imagine: You point your cellphone's camera at something and the image is whisked off to a remote server where some pattern recognition goes on. When the object is identified, the application will then respond with pertinent information. This could be encyclopedic-style data, a web link, a piece of music or a video, or anything else you can imagine.

So far the app is just a public beta in the U.K. and U.S., and it's apparently limited to recognizing movie posters, which will cause it to bring up the movie's traileron the phone. But the app's real potential is shown in the other info it will deliver about the movie: Nokia's press release says you can also "read reviews, and find the closest cinema where it is playing."--you can even buy tickets. By tying augmented reality into a location-based service on a cellphone, Nokia's creating an incredibly powerful tool that integrates cellphone cameras, GPS systems and mobile net technology. Exactly what kind of applications that tool could create has yet to be fully explored, so Nokia's approaching "businesses, content providers and agencies" to develop "innovative customized experiences" for their customers. Nokia's approaching this as a money-maker, of course, because it's possible to use Point and Find as an extremely targeted advertising portal--something the movie poster example amply demonstrates.

As well as recognizing movie posters, the system can also read barcodes and has a "category-specific text-entry search" option, so it may soon be possible to pick up an item in a shop, read its code and perform tasks like compare its pricing with other stores, and then command your phone's GPS to navigate you there. 

Point and Find's image recognition system remains remote because it's not possible to perform that sort of complex pattern recognition, comparing an image against a (potentially vast) database, on the limited tech inside current cellphones. But the sophistication of cellphones is improving, so Nokia's pointing the way to the future, and exploiting the business potential of augmented reality while it's still fresh tech. The system is so useful, however, that it'll likely end up on all our smartphones in the future and the database of digitally tagged images for recognition could even follow the Wikipedia model--essentially becoming a huge crowd-sourced database of extra information about many objects, people and places.

Read more about Augmented Reality.

[via Nokia

Add New Comment

5 Comments

  • Robert Rice

    Hey guys, thanks for the link Noah.

    "Augmented Reality" is beginning to be tossed around a lot, particularly in media by people that really have no idea what they are talking about and aren't interested in understanding it. The 3D media you see overlaid are not holograms by any stretch of the definition, but the visual appearance is close enough. The mass market understands what a hologram is. Examples like Star Trek's holodeck and the holographic doctor on Voyager reinforce the concept. As much as it pains me, the easiest way to describe AR to someone is to talk about holograms.

    The term Augmented Reality isn't great to start with...it isn't very euphonic and there are multiple definitions roaming around out there. Some are very specific and others are very broad. I favor the broader definitions, with multiple types (or levels or modes) based on the implementation.

    I've said that AR is not just about overlaying 3D graphics on a video stream. I think this has been sufficient in the past, but I think this needs to be expanded to include other elements that address who you are, where you are, what you are doing, what you are looking at, and what is around you. With this point of view, the Nokia application above certainly is augmented reality. Doing a basic image identification (comparing what the camera sees on the poster to a library of poster images) and then giving the user information (links, media, data) based on what is associated with the image in the database is one of the bare minimum very low level implementations.

    Of course, I'd rather be using real object recognition (not just image matching or identification) combined with immersive multimedia and wearable displays, but we still have to wait for that.

    Anyway, Richard is correct, this is not much more than image recognition (or more accurately image matching) but then, that isn't very far off from what everyone else is doing with fiducial markers (using a pattern for determining what media to display and where to display it) or the so called "markerless" methods that simply replace the pattern with an image (like a baseball card or a photograph). You don't need to display 3D content for it to be AR. Just give the user something meaningful and informative.

    Just my two cents : ) I'm looking forward to more dialog in the industry and at the conferences.

  • Richard Hurring

    Hey guys, thanks for your comments, Noah I loved the Rice article. We are in danger of getting caught up on semantics here, I do agree with the sentiment that this could be described as early AR, L zero in Rice speak. So my position was not to put AR down, almost the reverse, this image rec tech has been around a long time, it's great that Nokia are embedding it but I think it stops a long way short of even todays AR capability.

  • Kit Eaton

    @Richard. Noah beat me to it. If you're going for the :strict: version of AR, with live data-overlays on video or stills, then this isn't quite there. But machine vision is a key component in AR, and Nokia's system simply returns the relevant data in a non-overlay form. After all, running a movie trailer over the top of a photo of the poster just doesn't make sense.

  • Noah Zerkin

    This most certainly IS Augmented Reality. AR is a conceptual and technological paradigm. CV (computer vision) is a technology. It is one of many that will converge into Level 3 AR, as defined and described so well by Robert A. Rice, Jr. on his blog. http://curiousraven.squarespac...

  • Richard Hurring

    Guys, this is NOT augmented reality technology, I realise these are the latest buzz words in mobile but this is image recognition not AR. Richard Hurring