This AI Factory Boss Tells Robots And Humans How To Work Together

Siemens’s future “artificially intelligent” factory is aimed at a new trend in automation: telling robot and human workers how to help each other make things.

This AI Factory Boss Tells Robots And Humans How To Work Together

Robots are consistent, indefatigable workers, but they don’t improvise well. Changes on the assembly line require painstaking reprogramming by humans, making it hard to switch up what a factory produces. Now researchers at German industrial giant Siemens say they have a solution: a factory that uses AI to orchestrate the factory of the future, by both programming factory robots and handing out assignments to the humans working alongside them.


“Instead of programming [each robot], what we do is say, this is a machine that can do this [task], this is a machine that has the following capabilities,” says Florian Michahelles, who heads the Siemens Web of Things research group in Berkeley, CA. An AI program that Michahelles and his team have developed, called a “reasoner,” figures out the steps required to make a product, such as a chair; then it divides the assignments among machines based their capabilities, like how far a robotic arm can reach or how much weight it can lift.

Siemens is plugging into a larger trend in manufacturing, according to Mehdi Miremadi, a partner at management consulting firm McKinsey & Company. “For the first time, in the last few years, manufactures are seeing the real value impact of integrating the most sophisticated robotics and artificial intelligence in their production,” says Miremadi, who is unaffiliated with the Siemens project.

A McKinsey study he co-authored, “A Future That Works: Automation, Employment, And Productivity,” looked at 800 occupations and found that about half of the tasks workers do could be automated. But less than 5% of careers would be completely eliminated. In most cases, computers and robots would be picking up parts of people’s jobs. “People will need to continue working alongside machines to produce the growth in per capita GDP to which countries around the world aspire,” says the report. (McKinsey provides an interactive online tool showing that automation potential for these occupations.)

Siemens’s originally gave its automated factory project the badass Teutonic moniker “UberManufacturing.” They weren’t thinking of the German word connoting “superior,” however, but rather of the on-demand car service. Part of their vision is that automated factories can generate bids for specialty, limited-run manufacturing projects and compete for customers in an online marketplace. “You could say, ‘I want to build this stool,’ and whoever has machines that can do that can hand in a quote, and that was our analogy to Uber,” says Michahelles.

Now that Uber is no longer a venerated name, Siemens has rechristened the technology Click2Make to illustrate the point-and-click vision of a self-configuring factory. Michahelles and team have proved the technology can work on a small scale with a test system that uses just a few robots to make five types of furniture (like stools and tables), with four kinds of leg configurations, six color options, and three types of floor-protector pads, for a total of 360 possible products.


Programming Man As Well As Machine

“We also can include people,” says Michahelles. Like robots, human workers also get a description of capabilities and limitations that Siemens’s reasoner AI considers when making assignments, such as drilling a piece of wood or using screws to attach the legs of a stool.

The AI would also know some personal details about human employees, like if a worker is left- or right-handed, and what language they speak best. “The idea would be that, if we can describe all the machines and describe the skills of the workers, we are very flexible in arranging production flows,” says Michahelles.

The robot might hand parts, such as chair legs, to a human, he says. Then a human, with their finer dexterity, would assemble the parts. The reasoner would also enforce safety rules. “We have modeled the OSHA [Occupational Safety and Health Administration] work standards,” says Michahelles, providing an example. “If one item becomes too heavy for the worker to lift up, the robot would take over.”

“I think the human-robot interaction is the name of the game,” says Miremadi. “It will be the most important trend in the near- to mid-term…the next 5 to 15 years.”


One logistical—and safety—challenge of putting humans in the mix: Unlike robots that are bolted down, humans move around unpredictably. To deal with this, Siemens brought in a Microsoft Kinect camera system to identify workers and track their movements in three dimensions. That allows the reasoner to know who is available for work and how to position the robots so they can hand things back and forth safely, without smacking the humans around. (That’s a real danger. In 2015, for instance, a robot—not one made by Siemens—crushed and killed a technician at a Michigan auto-manufacturing facility.)

Related: Why It’s So Hard For Robots To Get A Grip

I ask Michahelles: Does this technology reduce humans to just another robot—one made of meat instead of metal?

“It should not be that way, because in that way we would just use an expensive human as an imprecise robot,” he says. The robots should serve the humans, doing the tasks that are boring, monotonous, or too physically demanding. “But when it comes to creativity and complex, intelligent tasks, this is where humans are superior,” he says. “The question is now, how can we build systems that combine strengths from both sides?”

Siemens customers won’t be starting from zero. Its factory robots (like those of its competitors) are already fitted with a program logic control device. “It’s basically a computer that’s telling the motors of the robot how to move and what to do,” says Michahelles. That provides a mechanism for the reasoner program to control the robots. And Siemens would provide profiles, called semantic descriptors, that tell the reasoner what each of its robot models is capable of. “So we [already] have technology, but we look for the problem,” says Michahelles.


That “problem” will be a company that sees an opportunity to make a lot of money by selling custom products. Michahelles gives the hypothetical example of a carmaker that can provide custom interiors as an upsell to buyers.

“It’s probably more customized and more expensive products, and also where there’s still a high part of human labor involved,” he says. But as a researcher, Michahelles has no control over how Siemens pitches new technology to customers. “The problem of disruptive innovation is, as long as they can sell the stuff they have, why would they risk something unproven?” he says.

There has been some progress toward developing pilot projects with customers Michahelles says, but “none I would be ready to publicly share.”

The Robots Are Becoming Handier

Although Click2Make integrates humans and robots, the overall trend in manufacturing will be for the number of humans to continue shrinking, according to Miremadi and his colleagues at McKinsey. They predict that up to 60% of factory tasks in the U.S. could be automated, for instance, though the transition will take decades. McKinsey’s report assumes that there will still be growth in jobs; but, as with automation in the past, the types of jobs people take will change.

One major advance in factories is the growth of biomimetic hands that closely match those of humans’. “The hands of the robots on the factory floor are becoming more sophisticated, so naturally they are doing more and more complex actions than what we are used to seeing,” Miremadi says. The closer interaction of humans and robots in factories will accelerate the process, he says, with humans training their replacements. Using a method called kinesthetic learning, people can teach certain dexterous robots, such as welding bots, complex maneuvers by manually directing a robotic arm or hand through a motion several times.


Programming robots used to be a job for engineers. Now advanced learning algorithms allow machines to pick up skills from workers on the assembly line. Robots are even gaining the ability to improvise slightly. Robots that learn to identify and pick up a particular object are increasingly able to identify and handle items with similar features, color, shape, and other attributes.

Related: A New Point And Click Revolution Brings AI To The Masses

“More and more [manufacturers] are becoming comfortable with having robots in essence running not just one specific activity but a set of activities,” says Miremadi. “And [humans] in essence play more of a quality check, monitor role—the managerial role versus doing-the-activity role.”

“There will always be tasks that a machine can’t do because it’s just impossible or too difficult or maybe just too expensive [to engineer],” says Michahelles. “We can carve out the expensive part for the human and have the other part done by the machines.” He mentions, for example, having to climb underneath a car chassis to attach components as being very difficult for a robot to do, but then adds, “this will change eventually.”

In the long run, humans will have to progress to more creative, intellectual work; they can’t count on their current advantage in dexterity lasting, says Michahelles. Otherwise, “it’s just a race against time, where the human will lose.”

About the author

Sean Captain is a Bay Area technology, science, and policy journalist. Follow him on Twitter @seancaptain.