"If you're watching TV and you hear a helicopter in your surround sound," says Santiago Alfaro, a grad student at MIT's media lab. "Wouldn't it be cool to just turn around and be able to see that helicopter as it goes into the screen?" Sure would. And Alfaro has realized that idea, in his thesis project, Surround Vision.
The set-up was actually fairly simple. The basic components were a TV, a hand-held monitor with a magnetometer (like a compass--found in newer smartphones, it measures both direction and strength of magnetic fields), and video footage of a scene taken from multiple angles at once. Alfaro was then linked all those together. The TV shows footage taken from a central camera. And if the handheld is pointed at the TV, that's what it shows as well. But if you move the handheld around, the magnetometer registers where it's being pointed--and in turn, the screen shows video taken from the corresponding camera angle.
Why would you want this, and why wouldn't you simply build a huge, panoramic TV? Several reasons, actually. For starters, a huge TV isn't feasible cost-wise, or space-wise.
But more importantly, Alfaro's design is meant to turn TV watching into something interactive and unique to the individual. For example, you can imagine an immersive episode of Lost, where you watch with friends and everyone in the room is exploring a different portion of the scene playing out on the central screen. You could even imagine watching a basketball game from a camera angle you choose in real time--or just watching the players mugging on the sideline.
TV thus transforms from being a passive, one-way experience to something that's personalized and visceral--something akin to virtual reality.
Alfaro's invention could also readily be translated to iPads or smartphones like the iPhone 3G S, Motorola Droid, or Nexus One since those have magnetometers in them. The only thing you'd need extra would be a few more cameras on site--something that's potentially cheap to do, in the age of digital video.