Kinect TV And Sesame Street Hack The Next Generation Of TV

Xbox Kinect TV plans to bring interactive, immersive experiences to live action television and children’s books with the help of National Geographic and Sesame Street’s Workshop.

Kinect TV And Sesame Street Hack The Next Generation Of TV


Xbox is unveiling a sharp idea for the next generation of television: interactive, live-action content, produced in partnership Sesame Workshop and National Geographic. Downloadable, linear episodes run like a normal television show but give children opportunities to play simple games with familiar characters and don virtual costumes that mold to their bodies and play around with the show’s environment. A series of interactive children’s books is also in the works. Dubbed “Project Columbia,” they allows burgeoning readers to explore the otherwise static world of a picture books with games, sounds, and augmented reality.

“An analog television show is not designed as a one-on-one experience,” Dr. Rosemarie Truglio, Vice President of Education and Research and Sesame Workshop, tells Fast Company. With an immersive environment, fluffy characters tailor the learning experience to different skill levels and invite children to explore imaginary and exotic locations.

A TV/Video Game Hybrid


Microsoft views Kinect TV as a Goldielocks compromise between the relaxed experience of television and the cognitive demand of a video game. “There are no idle states,” says Dave McCarthy, Xbox General Manager for Kids and Lifestyle Entertainment. If children are intent on being couch potatoes, the show proceeds as normal (but not without some humorous prodding to play along). Thus, children interact with the live-action, analog version of the characters, not an animated replica.

“We collaborate right at the kind of script and theme creation level,” notes McCarthy. “Right on the set, we will shoot a bunch of different outcomes depending on how the child reacts.”

The characters’ environment, too, becomes a virtual playground, where children are encouraged to identify shapes, count objects, and discovery hidden items. In the Kinect National Geographic episodes, children can explore wilderness environments as the animal itself, foraging for food in a animal-wrapped avatar that molds to their bodies.

The experience, Microsoft hopes, is intuitive enough for families who want a game-like experience without the difficulty of learning rules, dealing with frustrating win-lose states, and managing progression through an extended virtual world. Sometimes “we just want to chill out and watch TV,” says McCarthy. Given that children do still watch TV, Microsoft wants to place a product where children still naturally spend their time and will gently invite them to play with Kinect at some point during the experience.

Seriously Playful

For decades, Sesame Street’s research-tested call-and-response system has been haunted by an elephant in the room: When a character like Snuffleupagus asks children how many peanuts he’s eating, there’s no guarantee that children actually call out the right answer. With Kinect “If they get a wrong answer, we give them these helpful hints, so that they’re getting more information,” says Truglio. So-called “scaffold learning” pairs discovery with direct instruction and hints, to give children the freedom to experiment, but not so much leeway that they quit from too many failed attempts, or inadvertently end up creating false perceptions of the world.


Additionally, as every parent knows, children seem hard-wired for kinesthetic learning. They grab, prod, slam, and toss every item they can get their tiny little hands on. “Young children, they learn through play,” says Truglio. “They need to be able to manipulate things with their hands, they need to use their bodies.” Whether it’s counting objects while tossing coconuts or physically reaching out to three-sided figures to identify a triangle, the immersive experience helps imprint lessons in a way that transfers to the outside world.

Sesame Street 2.0 also hopes to prepare children for the new media world. “Children are born into this media-saturated world,” says Truglio. Yet, “a young child is often excluded from new media,” she says, noting that tiny-buttoned controllers and keyboards require too much dexterity for the still-forming fingers of very young children.

Controllers have also, by default, excluded parents, who are uncomfortable with button-mashing themselves. “Parents need an opportunity to be more playful with their children,” argues Truglio, who has previously talked with Fast Company about the vital importance making games inviting for non-gamer parents to become involved in the video game playing of children.

The final launch product, still in early testing, is “Project Columbia,” Xbox’s interactive children’s book. Unlike static words on a page, Project Columbia helps children understand concepts by playing with them in an immersive environment. “We make sense of the world best when we have a reference in the world that relates to our previous experience. And, those previous experiences are deeply tied to our body,” says Alex Games, Xbox Education Design Director. For instance, he says “near and far only make sense with reference to where you position yourself in the scene.”


In the Project Columbia promotional video above, children learn superlatives and the importance of chores simultaneously, seeing a room go from messy, to messier, to messiest, and then understand the meaning of each degree by physically moving their bodies to clean up the environment.

Kinect Sesame Street and Kinect National Geographic will be available for purchase next spring with a “season’s worth of content,” says McCarthy. Project Columbia is being targeted for the next holiday. Xbox also plans to partner with Pixar (for a game called Rush) and Double Fine productions (for Happy Action Theater) for similar interactive, immersive TV experiences for the holiday season and later in 2012.

McCarthy says that Xbox envisions using interactive technology for all sorts of traditional children’s products, including toys. At E3 this year, Fast Company got a sneak peek at Kinect technology that transforms camera snapshots of physical toys into a digital version in the Kinect environment, which replicates a pretty accurate version of color, shape, and size.

Indeed, for some children, analog objects are simply broken computers, as adorably demonstrated by this child, who treats the magazine like a broken iPad:


Follow Greg Ferenstein on TwitterGoogle+, or Facebook. Also, follow Fast Company on Twitter.

About the author

I am a writer and an educator. As a writer, I investigate how technology is shaping education, politics, Generation Y, social good, and the media industry.