Robots make thousands of tiny decisions a second, but they don’t emote or verbally communicate. So when a Roomba turns 90 degrees in the middle of the room, we’re only left wondering why and if something is broken.
Now, two researchers at the MIT’s Aerospace Controls Lab–postdoc Ali-akbar Agha-mohammadi and graduate student Shayegan Omidshafiei–are creating a system to peer inside the minds of autonomous robots. “Measurable virtual reality,” as the researchers call it, can visualize what robots see and what they’re trying to do.
For their experiment, they set up a motion-tracking system, coupled with a projection system, that can be installed in any room. The floor glows like a big game of Settlers of Catan, with various tiles signifying terrain like rocks, grass, or roads. Then, as drones drive or fly over the board, they make decisions just like they might in the real world–“fly around those mountains,” “drive down that street.”
As the robots make decisions, the projectors step in to visualize what those decisions look like. In one instance, three drones fly with radars in their shadows, demonstrating just how far they can “see.” In another, a drone drives up to second drone at a fork in the road, fires out a beam, and coordinates which robot will drive down which street.
These are by no means advanced, or data-dense visualizations. And outside this lab experiment, the visualizations couldn’t really scale (unless you plan to map the entire earth with a giant projector). But I’d argue that they are still good and important design. Because every time we walk on the sidewalk, we’re offered countless visual cues contextualizing the people around us. Are they walking at a brisk or leisurely pace? Are they smiling or scowling? Are they dressed for work or are they dressed for a club? Such indicators may lead us to make all sorts of wrong assumptions, but it grounds us in the ever-shifting social ecosystem of our lives. And by contrast, a world in which drones go about their day, completely pokerfaced and detached from humanity, is a world that’s no longer designed for the human experience. It’s streamlined for robots.