AI researchers are creating ‘Minecraft’ structures that build themselves

When a group of AI researchers started using Minecraft to simulate cellular growth, even they were startled by the sophistication of the structures—such as a jungle temple with a fully-functioning arrow trap—they were able to create.

Their goal was to build complex Minecraft structures by having each block learn to communicate with the ones around it, mimicking how the human body develops from a single cell in a process called morphogenesis. That model worked out even better than they anticipated, with one block growing into exactly the kinds of objects it was trained to create. In addition to the jungle temple, the system generated immaculate castles, stylish apartments, and even a caterpillar that regenerates after being cut in half.

Sebastian Risi, an AI researcher at IT University Copenhagen, says this work could be a foundation for more ambitious projects to come, including a version of Minecraft that simulates evolution. He conducted the research along with his ITU Copenhagen colleagues Shyam Sudhakaran, Djordje Grbic, Siyan Li, and Elias Najarro; University of York’s Adam Katona; and Shanghai Jiao Tong University’s Claire Glanois.)

“We have the basic components now,” Risi says. “It’s just about figuring out how to connect them all.”

From one cell to another

The Minecraft project’s underlying concept, called cellular automation, has been around since the 1940s. It’s the idea that cells in a system can live, die, or reproduce according to a set of rules. John Conway’s Game of Life, which dates to 1970, is probably the best-known example.

But in those early systems, the researchers had to program the rules themselves, a process that eventually becomes impractical as the systems become more complex.

This new Minecraft simulation works differently. Instead of coding the rules by hand, the researchers used neural networks to train each cell on what a set of finished structures should look like. Each cell figures out what type of Minecraft block to become by looking at the cells around it, then passes that information along to its neighboring cells so they in turn can figure out what to become next.

“We only told it what it has to grow, but we didn’t tell it exactly how it has to grow it,” Risi says.

Excited to share our work on Morphogenesis in Minecraft! We show that neural cellular automata can learn to grow not only complex 3D artifacts with over 3,000 blocks but also functional Minecraft machines that can regenerate when cut in half 🐛🔪=🐛🐛

PDF:https://t.co/hi573xzWIG pic.twitter.com/m19572pcIe

— Sebastian Risi (@risi1979) March 17, 2021

This type of intercellular communication roughly approximates the way in which human cells work together toward a common goal, which Risi says is part of what makes the research exciting. The exact language our cells communicate is still somewhat of a mystery; Risi hopes that by visualizing the messages that each cell sends—something the researchers hope to do in the future—they can help biologists understand more about how the body works.

“It’s difficult to study those things in nature because it’s hard to extract those exact messages that cells pass,” he says. “We think you could get some insights about the nature of what cells need to send in order to self-assemble.”

Biological building blocks

Even if this project doesn’t unlock the secrets of the human body, it could still lead to some other interesting applications.

For one thing, it could create new possibilities for Minecraft as a game. You might imagine a game world with structures that self-assemble before your eyes, or that heal themselves automatically after being destroyed. Perhaps players could also get involved in the creation process by starting to build a structure and then letting cellular automation take over.

“Instead of having a fixed world, as you explore the game, the world could grow around you or grow with you, maybe also involving you in this kind of cellular automata,” Risi says.

“We saw with this caterpillar, you cut it in half and get another caterpillar,” he says. “Now we need to figure out how to build this evolutionary loop, so that starting from this caterpillar, other organisms can evolve.”

The researchers’ paper throws out some wilder ideas as well, like how further research in Minecraft could help us figure out how to create better artificial organs, or real-world buildings that repair themselves. Risi acknowledges that this sounds like science fiction, but the researchers are serious about taking their lessons from Microsoft’s virtual world out into the real one.

“It’s cool to do stuff in Minecraft,” he says, “but it goes beyond Minecraft, in that we want to ultimately use it as a tool for developmental biology.”