This Device Creates A 3-D Soundscape To Help Blind People Navigate Through Cities

Pulling location data from streetlights, stores, and buses, it translates them into sound.

For someone who can’t see, meeting a friend on the other side of town can mean step after step of stressful navigation: Finding a bus stop, figuring out what’s going on if the bus is delayed, getting through the gates at the subway, and countless other obstacles along the way. A short trip might take hours of planning, and even familiar routes, like the path to the closest grocery store, can be so challenging that some visually impaired people rarely leave home.


While a smartphone can help, a new device from Microsoft goes much farther, creating a “3-D soundscape” around the person wearing it. A small headset worn just above the ears conducts sound through bones, so it doesn’t disrupt hearing as the wearer walks. A small clicking sound tells someone they’re heading straight ahead, and pings let them know if they’re veering into the street. If they pass something interesting–like a new coffeeshop on the left–the device points it out.

Unlike smartphone maps, which rely on GPS or Wi-Fi triangulation to navigate and are often slightly wrong, the device is designed to pull data from actual physical beacons on streetlights, stores, moving buses, or other objects. Then it combines that with information from databases, like bus schedules.

“A beacon can say ‘Okay, you’re standing at the bus stop right now, and you’re standing in exactly the right place,'” explains Dan Hill, executive director of Future Cities Catapult, an urban innovation organization that collaborated with Microsoft and mobility organization Guide Dogs as they built the prototype. “Then it can say the bus is going to be here in 10 minutes.”

The device also goes beyond simple navigation to explain what someone is passing. “It creates a sort of sense of spontaneity that those of us who are sighted have,” says Hill. “If I’m going to the shop, I might just duck in an alleyway to check out an interesting gallery in the corner. It’s easy for me to do that, whereas a visually impaired person has to plan everything so carefully they can’t just do that.”

The group tested the prototype in the U.K., on a route from Reading to London that involved catching a bus, a train, and even testing the device in a grocery store, where it can be used to read barcodes off the shelf.

“The trials of the technology show that it can reduce the anxiety and stress associated with making a journey in a town or city,” says Jenny Cook, head of strategy and research at Guide Dogs. “62% of participants reported an increased feeling of safety, confidence and resilience, allowing them to relax into the journey.”


Though the technology is still a prototype, Microsoft is figuring out to pursue it, including ways that it could be used by those who can see. “How do you design for the 1% and create novel innovation for the 99%?” says Dave Campbell, chief technology officer for Microsoft’s Cloud and Enterprise group. “We’re absolutely looking at the universal applicability of it.”

As this type of wearable technology develops, it may also start to reshape how cities and buildings are designed. For example, if we’re all wearing devices with our train ticket information in 15 years, train stations may no longer need ticket barriers, for example.

“Right now we’re making design decisions at places like train stations based on today’s technologies,” says Hill from Future Cities Catapult. “In our agency we work very closely with urban planners, who traditionally haven’t taken this more sort of tech enabled approach. Something like this could radically change the way stations are laid out.”

For Hill, the project also serves as a clear example of what a smart city might look like. “We’re interested in making something tangible and understandable and communicable,” says Hill. “There’s been a lot of talk around smart cities, and the way technology might change the city. But a lot of it’s been abstract and hasn’t really communicated is that a real thing, or what could really happen. We’re trying to make it real for people.”

About the author

Adele Peters is a staff writer at Fast Company who focuses on solutions to some of the world's largest problems, from climate change to homelessness. Previously, she worked with GOOD, BioLite, and the Sustainable Products and Solutions program at UC Berkeley.