With Response To Touch, ToyTalk’s Conversational Characters Get More Realistic

In a new app called SpeakaLegend, kids can play with their favorite characters more like they play with their friends.

With Response To Touch, ToyTalk’s Conversational Characters Get More Realistic
Characters from SpeakaLegend App [Illustrations: courtesy of SpeakaLegend]

“Who else have you made friends with?” an animated Cyclops on ToyTalk CEO Oren Jacob’s iPad asks.


“I’ve been friends with earlier today the dinosaur and the dragon and the bigfoot,” Jacob tells him.

“Oh, he’s not so big,” grunts the Cyclops. “Nuh uh, I am bigger than him, but nobody calls me big eye or anything. Why is that do you think? Why does nobody call me big eye?”

“Maybe they feel you’re self-conscious about it or that you have a concern that you don’t have two eyes?”

“Well that’s okay, I don’t need no fancy nickname or nothing. Just Cyclops.”


Cyclops is part of a new app from ToyTalk called SpeakaLegend. Like the company’s first two apps–a participatory gameshow hosted by an enthusiastic yellow blob named Winston and a zoo where kids can talk with animals called SpeakaZoo–kids can converse with the characters inside SpeakaLegend, who respond to them, in character, with hundreds of pre-recorded lines. What makes SpeakaLegend different than its predecesors is that its characters and scenes also respond to touch.

When Jacob nudges Cyclops’s foot, the giant jumps. When Jacob talks with the Jackalope, another of the ten mythical creatures included in the game, he can set dustballs flying, play with scorpions, or feed her carrots. A dragon with a cold will sneeze fire if you touch her nose. Touch, Jacob says, is just another dimension of realistic conversation. “It’s a pattern we regularly do,” he says. “You’re talking to somoene and eating with your hands. Or kids are playing with their train on the floor, talking about the train back and forth.”

Though it might be natural in life, touch in tandem with conversation is not exactly easy on a tablet computer. ToyTalk had to consider design questions like how a conversation changes if a child touches a character who is talking–in real life, if I suddenly grab your arm, you might pause and ask what I want–or if a child tries to talk to Jackalope while feeding her simultaneously.

Each lobbed carrot and rolling dustball has its own sound effect. That needs to be balanced with characters’ lines and kids speaking into the microphone, which can’t be used at the same time as the device’s speakers. ToyTalk orchestrates all of this, as well as speech recognition, natural language processing, and selection of dialogue, in real-time.

One benefit of adding touch into an already complicated process, Jacob says, is that SpeakaLegend can be more of a game than either of its audio-only predecessors. As kids spend time playing with and talking to one of the apps’ characters, they earn progress in a “happiness bar.” They need to fill all nine characters’ happiness bars in order to unlock a mystery character (hints from Fast Company’s short interviews with Cyclops and Jackalope strongly suggest it is a unicorn).

SpeakaLegend costs $2.99 and is ToyTalk’s first paid app.


ToyTalk doesn’t serve advertising within its apps to comply with COPPA laws (and also, Jacob says, “Philosophical, religious, and business” beliefs). It does, however, have a couple of other promising lines of business.

Jacob and his co-founder, Martin Reddy, decided to start ToyTalk after Jacob’s nine-year-old daughter asked him if she could Skype with her American Girl doll. Unlike most parents who might be asked that question, Jacob had a background that allowed him to think seriously consider making that happen: He and Martin helped rebuild Pixar’s internal film-making software (Jacob was Director of Studio Tools and Martin was Head of Engineering). They solved the “Skype with fictional characters” problem by creating a conversational authoring platform called PullString. The software is what would happen, Jacob says, “if Microsoft Word, Final Draft Pro, a hierarchy browser, Google Search, and a chat bot had a baby.” It’s basically a tool that creative teams can use to create and manage dialogue possibilities. Given that many game studios still use an Excel spreadsheet to keep track of character dialogue, this software is marketable. ToyTalk wants to license the technology to entertainment companies who want to make their characters conversational.

ToyTalk plans to use the 20 million snippets of children’s voices it has collected since launching its first app to create a voice recognition engine specifically designed to understand children. That could be used not just to create conversational entertainment, but also in language, reading, and pronunciation apps.

The company’s technology could help power a shift toward apps that allow kids to shape experiences with the characters on their iPads. And it won’t necessarily be just voice and touch that these new characters understand. “How you look how you sit, how you pose yourself, and the voice you use, are all involved in communicating human to human,” Jacob says. “As technology gets more advanced and we can teak advantage of these things. The higher fidelity signal that I can get from how you’re enjoying something, or not, speaks to the truth of conversation.”

About the author

Sarah Kessler is a senior writer at Fast Company, where she writes about the on-demand/gig/sharing "economies" and the future of work.