For Breakfast, This Robot Will Make You The Pancakes It’s Learning How To Cook From Reading WikiHow

In a lab in northern Germany, a robot named PR2 is learning to make pancakes and pizza by browsing Internet recipes.

It’s not exactly easy, even though the instructions sound straightforward to a human. “An instruction might state ‘flip the pancake,’ but the robot needs to infer that it needs a spatula to do that,” says Michael Beetz, head of the Artificial Intelligence Institute at the University of Bremen, where researchers are helping PR2 figure out how to follow the recipe.

A long list of other details are missing: How to hold a spatula, how to turn it, how high to flip the pancake. PR doesn’t know “what might go wrong and why,” Beetz says. “For example, you might damage the pancake if you push too hard.”

So Beetz’s lab is filling in the gaps, crowdsourcing interpretations of the instructions via humans using Amazon’s Mechanical Turk platform. They also set up a virtual reality game, so PR2 could watch a player perform each step and learn what to do.

The project, called RoboHow, is a step towards getting to robots that can follow human instructions, instead of needing to be programmed for each specific task. The team is building a “robotics knowledge service” that our future robot butlers can access in the cloud–and eventually, in theory, they’ll be able not only to make us breakfast, but anything else we ask for.AP