“Watching the world through a computer screen is agonizing,” OneZero editor Sarah Kessler recently wrote about working from home during the COVID-19 pandemic. Having worked from home for the past 15 years, I know exactly what she means. The screen world is its own place: A world filled with a constant onslaught of too much bad news, too many work demands, and an ever-dwindling number of friends who are willing to hear me complain about it. Even on a great day, the pixels are a poor simulacrum for real life—for actually tasting a great meal instead of looking at one on Instagram, or laughing with friends on a couch instead of through emoji.
Technologists didn’t always imagine the connected world looking like this, ever trapped behind a glass touchscreen (a notable germ haven, incidentally). For over two decades, researchers seriously imagined a different approach to computing: tangible interfaces. These are interfaces that have broken out of the glass and live alongside us in our real environments, allowing us to connect with people far away through touch or even smell. In a world under quarantine, where all of us are feeling a little isolated, it’s an idea that we need to bring back.
A means for human touch
Tangible interfaces offer the promise of reaching out to literally touch something or someone. In 2013, Disney Research imagined how air itself could become tactile, allowing you to feel objects and even textures through simple, coordinated puffs of air. We have visually immersive VR headsets in 2020 that are superb at tracking the specific movements of our body, but we still have nothing that comes close to offering us a sense of touch.
MIT’s Tangible Media Group, the mecca of this work, introduced a system called ZeroN, which allowed you to control a computer by grabbing metal balls that floated in midair. You could literally place a ball in space and leave it suspended. It’s easy to imagine how such a device could be networked, with more components, allowing two people, each with their own display, to remotely manipulate a shared object. Then in 2013, the group introduced inFORM, which used dozens of hydraulic pegs to create a 3D display of moving sculptures. You could literally play catch with someone through inFORM, tossing a ball into their hands. Having seen inFORM with my own eyes, I can only say it’s every bit as, if not more, unbelievable in person. The mind-melting GIFs barely do the technology justice.
The road less traveled
But tangible interfaces have always been a somewhat esoteric realm of the technology world. “[Tangible interface] is the road less traveled. Well-researched until the early 2000s, and then . . . not so much,” says Matt Webb, founder at Job Garden who, in a previous life, ran one of the most cutting-edge design studios in the world, Berg.
A personal favorite project from the era was Dangling String by artist Natalie Jeremijenko. It was a simple, 8-foot-long piece of plastic wire hanging on a small motor from the ceiling at Xerox PARC—the corporate research lab that brought us technologies like the computer mouse—but it demonstrates the potential of tangible interfaces superbly.
As invisible network traffic increased in the office, the string spun faster—a translation of bits into our environment that anyone could grasp and even enjoy. As Jeremijenko told me when we spoke in 2016, the idea was so radical to some of her peers who were obsessed with traditional graphical expressions of data that she broke into the lab in the middle of the night and crawled into the ceiling tiles to set it up. Her own colleagues who helped with the project begged for anonymity, “because it didn’t look like the [Xerox PARC] kind of geek, it looked like a little dangling string in the corner,” she said. “It didn’t look like the real stuff from this high-tech corporate think tank.”
Within a day, though, the office quickly grasped the correlation between the whir of the string and the activity of the network. Then they would instinctually poke their heads out of their offices, see it spinning, and yell about somebody hogging all the network bandwidth. The simple plastic string became an office tool. At one point, a hacker was trying to break into Xerox PARC’s system. How did they know? The string kept going on and off at strange intervals when Xerox PARC researcher Mark Weiser was working alone at the office into the middle of the night. There was no one else around to be using anything.
A calmer solution to work life
The string is a perfect example of one benefit of tangible interfaces: They’re what Weiser called “calm technology.” Unlike that glowing red dot notification on your desktop, calm technology can sit in your periphery as the sort of thing you notice without noticing it. This is the possibility of computing that’s truly built into our environment. Sound oversimplified or too good to be true? Then consider when you walk outside, are you overwhelmed by birds chirping, the scent of freshly mowed grass, and thousands of leaves blowing in a breeze? All of this is rich information, but it’s offloaded into the world of the visceral rather than the digital. It’s easier for us to manage information in the real world.
“We don’t just think with our conscious mind, our ‘focus.’ We’ve also got unconscious layers beneath that which we can give partial attention to, and a broader ‘peripheral awareness’ base to the pyramid. When we work, we’re smoothly moving objects up and down that pyramid,” explains Webb, who wrote a book on this topic. “In the real world, all of this is perfectly natural. But on computers, and particularly on small screens like phones, it’s hard to get into that flow. So we try to replicate it with snoozed emails (pushing a task down the pyramid to have it come back later), or to-do lists, or sitting in a closed meeting room to focus.”
The screen is an imperfect solution to the problem of productivity, and over the years, Webb and his team imagined alternatives. One example is the Availabot, from 2006. It was a little USB mannequin that plugged into a computer. When you were available on AIM—remember AIM??—he stood firmly at attention. When you went away, he slumped like a puppet whose strings had been cut.
Availabot is an adorable, low-stress way of conveying digital presence in your environment. Another demo he built was Bumptunes, which allowed you to skip ahead in a track by slapping your laptop like a jukebox. Berg also developed the Little Printer, a networked receipt printer that transformed all sorts of tasks and notes into a tiny newspaper.
When I suggest to Webb that the iPhone, which launched in 2007, killed tangible interfaces, he argues that’s too simple. “It’s not just that smartphones happened. It’s that design has become something that has to be quantified and explainable,” he says—no doubt alluding to the massive data tracking behind modern apps, which allows designers to quantify scrolling and tapping statistics, as well as user conversion rates to buy items or subscriptions. “The last 20 years has been a story of new users, people using technology for the first time, so everything has to be clear simply by looking at it.”
In real products
Tangible interfaces offer a tantalizing vision of computing, one that is both more intimate and less demanding of our time than existing interfaces. But just about all the tangible interfaces we’ve seen have been prototypes; they are impressive but crude teases of what could be if designers and companies continued their old obsession with physical interfaces. (Microsoft considered building the Xbox One with controllers that emitted scents, giving games an olfactory component, before abandoning the idea!)
The most notable experiment I can remember in this field that actually shipped—beyond a few networked intimacy toys—arrived from Apple as a small part of a major product, with the introduction of the Apple Watch. Called Digital Touch, you could draw, tap, or even share your heartbeat with another Apple Watch, or even iPhone and iPad, user.
Digital Touch isn’t the reason anyone I know bought an Apple Watch, and I haven’t heard the tool referenced in years. But this is a tangible interface at its weirdest and finest. Apple’s wearable computer became a portal to send and receive physical sensations. And as many of us hunker down for the next weeks or months, craving more contact, it’s hard to imagine a field of UI research more important than this one.