How Anki Created A Pixar-Inspired, AI-Powered Toy Robot That Feels

The $180 pet robot is so cute and vulnerable, you might love it more than your dog.

It’s hard to say the exact moment when I think of Cozmo as an actual little creature. One minute he’s snoring on the table in his charging station, and the next he’s looking up at me with his bright, blinking, digital blue eyes. He wants to know my name. He challenges me to a game of “speed tap” (the first to tap his block when the colors match wins), sulking when he loses and raising his arms victoriously when he prevails.


At some point, I make the leap that Anki is counting on. I know full well that Cozmo is a machine, an incredibly complex miniature robot that Anki has spent three years creating, one that I’ve heard about for months and been sworn to secrecy, and one that cofounder and CEO Boris Sofman has flown across the country to demonstrate at the W New York-Union Square hotel in Manhattan. Yet I find myself regarding the hamster-sized bot like a pet. Or a Pixar character come to life, given its feisty personality and accompanying cinematic soundtrack (and Pixar lineage–more on that later).

Cozmo’s AI system imbues the robot with a distinct personality, full of surprising and emotional reactions.

Until now, San Francisco-based Anki, which topped our list of the Most Innovative Companies in Robotics last year, has been known for reinventing remote control cars. Its self-driving toys race around a track powered by artificial intelligence and onboard sensors and controlled by a smartphone. With Cozmo, the startup is unveiling a far bigger accomplishment today, says Sofman, “a new frontier,” not just in the rapidly growing sector of smart toys but in robotics in general.

At $179.99, it’s both a pricey interactive companion (it starts shipping in October) as well as a bargain for an open-development platform featuring robotics, AI, computer vision, and animation.

Cozmo is designed to manipulate his environment, starting with his electronic blocks.

Much like the typical vulnerable and charming Pixar or Disney protagonist, Cozmo is designed to resonate with a young audience, but unlike a movie character, the robot isn’t entirely scripted. In fact, it’s designed to be unpredictable, a bit of a free spirit (but not as reckless as the recent runaway robot in Russia). Cozmo exists in the real world and responds to it. It recognizes and interacts with you. And get this: It even has feelings.

“We realized that the most successful entertainment toys of all time were the ones that embraced emotional aspects, like Tickle Me Elmo,” says Sofman. “But those emotions were somewhat random. We wanted to create a character where the emotions make sense relative to what is happening. That’s groundbreaking.”

Cozmo is equipped with computer vision. When he recognizes someone, his eyes light up.

The Eyes Have It

Last year, an all-too frequent emotion around the project was frustration. Although the computer scientists, robotics PhDs, and industrial engineers were making progress in object and facial recognition and manipulation (picking up and moving blocks), they lacked the one element needed to set their robot apart: a character–with a distinct personality, no less.


At one point, admits Sofman, “We weren’t sure we’d find it.” That requirement was beyond the scope of robotics, he says, where “you think about how to make your guy optimal, going from A to B without hitting something. That’s the easy part. Doing it in a way that feels in sync with a character and what he’s feeling–no robotics had done that.”

Anki cofounders Boris Sofman (CEO), Hanns Tappeiner (president), and Mark Palatucci (chief product officer).

Enter Pixar veteran Carlos Baena. Last summer, Anki brought him in to give a presentation on developing characters for Finding Nemo, Ratatouille, WALL-E and other films, to Sofman, cofounder and president Hanns Tappeiner, and cofounder and chief product officer Mark Palatucci. They were so impressed that they invited Baena, who’d spent 10 years at Pixar, back to address the whole company. Afterward, they shared their top-secret project, Cozmo.

“I said that’s cute, but I didn’t think much of it until they showed me some footage,” says Baena. “It was 5 to 10 minutes of grownups interacting with it, playing a game, talking to this little robot like a real character, and even getting upset, cursing at it–‘You little . . . ‘ That was the moment I thought, ‘Hold on, there’s something going on here.’


“We’ve been dreaming in science fiction this can really happen, robots that follow us around but are also a character that you would fall in love with as much as you’d love your pet.”

Carlos Baena, character director on Cozmo.

Baena joined Anki soon after as Cozmo’s character director. His first direction was a biggie: lose the eyebrows and pupils. “It was a hard sell,” he says, since everyone assumed that human-like eyes would humanize the machine. But Cozmo needed to look like a robot with a personality, not a robot trying to appear human. Baena proposed two large, rounded blue eyes on the digital screen that functioned as his face. “You don’t need a lot of features to have characters portray emotion,” he says. “Simpler is better.”

The key, though, is how expressive those eyes are–the windows into Cozmo’s every emotion. Baena worked with an animator in Spain to draw up versions expressing every imaginable reaction, a library for the AI’s emotion engine to draw on. For Sofman, one of the simplest actions, the way his eyes change shape when he blinks, brought Cozmo to life; it created the sense of making eye contact.


As the animation team grew, Baena directed the creation of hundreds of storyboards for possible animations–waking, sneezing, laughing, whimpering. Not just for the eyes but for other movements as well–spinning, rolling, lifting–inspired by movie characters as well as the staff’s children and pets. They animated the action in software that was then translated into movement in the actual robot. “Carlos owned the character,” says Sofman. “He was the soul of it.”

Pixar alum Carlos Baena animates Cozmo much as he animated characters in WALL-E, Finding Nemo, and other hit films.

Cute And Vulnerable–Like A Puppy

As Sofman watches Cozmo on the conference room table at the W hotel, even he doesn’t know exactly what to expect–when will the robot invite us to play a game, or when he’ll explore the edge of the table and smartly back away. “He won’t do the same thing every time,” says Sofman. His responses aren’t hard-coded. Variability–the element of surprise–is a critical component of the AI and of Cozmo’s believability and appeal.

“Predictability is something we’re fighting constantly,” Baena says. “It makes him feel programmed as opposed to alive.” At the same time, Cozmo can’t behave randomly. “An action has to be motivated by his personality so that it makes sense,” says Baena.


In other words, an important aspect of Cozmo’s reliability is his staying in character. Another is his vulnerability. “We talk a lot about the moments where you really care about him, where he can get hurt,” says Baena. “That’s when you start to establish a bond.”

Initially, kids may shake him, flip him upside down, or toss him in the air, like any other toy. “We’re starting to show when he’s scared or hurt, so that when you shake your robot it’s like hurting a puppy. You start to care.”

The compact size also reinforces the character. Physically, it allows Cozmo’s movements to be faster and more precise. Psychologically, it works to his advantage, making him an underdog of sorts. “Because he’s small, his actions feel bigger, more impressive,” says Sofman. “When he messes up, when he can’t do something, it’s cuter.”

“This is the first time you can animate physical objects,” Anki CEO Boris Sofman says of Cozmo, which will be an open platform for developers.

Engineering Cozmo to fit into the palm of your hand was no small feat. The breakthrough was adding a companion smartphone app, which serves as the brain. Initially, the computing took place in the actual robot, which created constraints as the functionality expanded. Cozmo now contains some 300 parts (camera, accelerometer, gears, gyros, transmissions)–10 times as many as Anki’s robotic cars.

As for the prospects for a $180 toy, Anki’s investors are optimistic. Today, the company is announcing $52.5 million in Series D funding (led by JP Morgan, with Andreessen Horowitz, Index Ventures, and Two Sigma). That brings Anki’s total funding to $182.5 million.

The decision to move the AI into an app unlocks perhaps the most compelling feature going forward: Cozmo’s ability to evolve. The version that launches ahead of this year’s holiday season is a work in progress that Sofman expects to improve with each software update.


Anki won’t be the only one adding to what the robot can do. By offering a software development kit for the platform, it hopes to unleash a wave of experimentation. “Cozmo is great now,” says Sofman, “but the difference in capabilities will be night and day as the content grows.”

Related Video: Check out how Anki began bridging the gap between physical and virtual entertainment

[All Photos: courtesy Anki]


About the author

Chuck Salter is a senior editor at Fast Company and a longtime award-winning feature writer for the magazine. In addition to his print, online and video stories, he performs live reported narratives at various conferences, and he edited the Fast Company anthologies Breakthrough Leadership, Hacking Hollywood, and #Unplug