Creating Virtual Interfaces for Physical People

Interaction designers have long been restricted to an input vocabulary of buttons and pointers. Now they’re gaining a complex and dynamic palette limited only by the physical capabilities of the human body.

I had a college science professor who was infamous for the phrase “The less tangible, the more concrete.” In other words, those factors in the world that seemed the most abstract–time, light, gravity–were actually the most real, consistent and measurable (although gravity always seemed pretty tangible to me).


I’m reminded of that phrase as I observe the advances in user interaction technology appearing throughout the electronics world. From touch and multi-touch phones and computers, to gestural interface gaming systems and radio transmitters embedded in everything from passports to pets, the connections between people and technology are becoming stronger, even as the medium for doing so becomes less visible and tangible. At the same time we have seen more sophisticated physical devices for interacting with the digital environment.


InterAction Labs SQWEEZE Game Controller

These tangible and intangible technologies have opened a rich world to interaction designers, who have largely been constrained to an input vocabulary of buttons and pointers. Now, designers are gaining a complex and dynamic palette built around (and limited only by) the physical capabilities of the human body.

This presents great opportunity but also risk. Most interaction designers lack a sufficient understanding of the ergonomics and kinesthetics of the human body. Moreover, there is a chasm in our understanding of how physical actions relate to information-based tasks. We learn complex physical interactions for physical activities: riding a bicycle, painting a picture, even cooking. But our physical interactions for information-based tasks tend to be relatively basic: turning a page, scrolling a wheel or pressing a button.

There are a handful of cases that apply physical dexterity for information-based tasks, but they are few and far between: using an abacus, telegraph, or more recently, texting & typing. But now, there are emerging solutions for leveraging the capabilities of the body to more effectively interact with information.


For example, a few weeks ago Clayton Miller posted a video of his 10/GUI concept (Fast Company’s coverage here). The video demonstrates the potential advantages of navigating within a desktop interface with up to ten fingers, rather than via a single cursor.

10/GUI is a well-thought out and clever approach to human-computer interaction. But it draws attention to the fine line designers will need to walk to effectively create physical human-computer interactions. Make the interactions too basic, and we’ve just replaced a mouse with a finger–convenient perhaps, but not necessarily an improvement. On the other hand, make the physical interactions too complex or demanding and run the risk of excluding people who don’t have the capabilities or proficiency to perform like a rock star.

Creating effective physical-to-digital interactions is a unique challenge. With the great human diversity in physical capabilities due to age, gender, physical condition and motivation, there is rarely a “one size fits all” solution. A human-centered approach involving the collaboration of interface & industrial designers with experts in human factors & ergonomics is necessary. Ultimately, with the continued spread of physical interaction technologies and metaphors, it is essential for designers to not only “know thy user,” but to know their users’ bodies.

Rob Tannen’s Designing for Humans blog

Browse blogs by more Expert Designers

Rob Tannen is an expert in designing products, interfaces and systems that accommodate the complexities of human behavior and capabilities. He has researched cockpit interfaces for U.S. Air Force, designed trading floor order systems for the New York Stock Exchange, and created touch screen applications for consumer appliances. Along the way he has developed a unique, multidisciplinary skill set for understanding how people interact with technology, which integrates expertise in ergonomics, ethnographic research, usability and information design. He has a PhD in human factors and is a Certified Professional Ergonomist.
Rob is Director of User Research and Interaction Design at Bresslergroup, the award-winning product development firm. He collaborates with designers and engineers on a range of consumer, medical and commercial projects. The firm specializes in interaction & industrial design, and mechanical innovation.


About the author

Rob Tannen is Director of Design Research & Strategy at Intuitive Company. He has a deep understanding of how to improve the connection between people and technology and has designed everything from mobile apps to complex financial systems to aircraft cockpit displays