advertisement
advertisement

For Google Maps’ trickiest challenges, AI is the answer

One new feature aims to steer drivers away from danger-prone roads. Another builds out maps with vital details for those who do their traveling by sidewalk.

For Google Maps’ trickiest challenges, AI is the answer
[Animation: courtesy of Google]
advertisement
advertisement

Every time you ask Google Maps to provide driving directions, it considers many options and selects one as the optimum route. Naturally, getting you to your destination in an efficient manner is a primary goal. But when you set out on a trip, efficiency isn’t the single most important factor. Above all, you’d like to get there safely.

advertisement
advertisement

That’s the premise behind a new feature that Google unveiled today during this year’s online version of its I/O developer conference. Google Maps will now identify road segments where drivers tend to slam on their brakes. It will try to route you around such areas even if they’re theoretically part of the most obvious route.

Figuring out where the danger zones are so you can avoid them is “one of the most complex problems I’ve been lucky enough to tackle in my time at Google,” says director of product Russell Dicker, who’s worked on Maps off and on for seven years. The company solved it by applying AI to data, as it’s been doing with a bevy of other recent and upcoming tweaks to the world’s most popular mapping app.

It’s pretty obvious why hard braking might be a sign of dangerous stretch of road: It’s evidence that drivers are reacting to something unexpected. And if everyone involved doesn’t react quickly enough, the result can be an accident. Indeed, Dicker says that the inspiration for the new Google Maps feature came from an incident a couple of years ago when a Google Maps product manager rear-ended his father’s car at “this intersection with one of those super-short yellow lights.” Everyone was okay, but the mishap led the Googler to delve into the topic of hard-braking incidents— the subject of considerable research by organizations such as the Virginia Tech Transportation Institute.

advertisement

We think that we’re going to have the ability to potentially eliminate around 100 million hard-braking events.”

Russell Dicker, Google
The more the Maps team looked into the issues that can lead to hard braking—which range from road geometry to sunlight hitting drivers in the eyes—the more comfortable it felt factoring them into its driving directions. “We’ve seen that there can be a sudden increase in hard-braking events along a segment when it’s raining extra hard,” says Dicker. “And so this was the next ‘Aha’ moment for us, because we realized that understanding environmental factors and helping people navigate them successfully was what Google Maps has done for years.”

So how does one identify roadways that are prone to hard-braking incidents? Google had an obvious opportunity to collect relevant data: The Google Maps app runs on smartphones equipped with accelerometers, allowing it to detect motion or the abrupt lack thereof. But phones aren’t bolted to vehicles; they’re subject to independent movement of their own within the cabin. That meant that raw accelerometer data was of limited value.

Google discovered a workaround in the fact that a decent chunk of Google Maps navigation involves Android Auto—the feature, built into many recent vehicles, that lets you project apps from your phone onto a dashboard touchscreen. A phone that’s powering an Android Auto session is at least tethered to the vehicle it’s in, and Google found that it provided more robust evidence of hard braking. By using this data to build a machine-learning model and then extrapolating for the larger data set of motion data from Google Maps apps on phones inside other vehicles, the company was able to build a database of hard braking-prone areas that meant something.

advertisement

“There’s no one place where half the time you drive there, you’re going to have a hard-braking event,” says Dicker. “The world isn’t built like that. And so it’s [about] trying to find the really subtle things. That’s why the large amount of data is super helpful, because we can really try to detect those patterns.”

The data resulting from this AI-infused process is historical in nature rather than in real time. That means that it’s not going to help people avoid accidents in progress—something Google Maps already does through live traffic data. But it can steer drivers away from problem spots whose risks may haven’t even been readily apparent until now. And given that Google Maps has more than a billion users, the cumulative effect could be profound.

“We think that we’re going to have the ability to potentially eliminate around 100 million hard-braking events on routes driven with Google Maps every year,” says Dicker.

advertisement

For those on foot

Along with motorists, another important Google Maps constituency is pedestrians and others traveling primarily by means of sidewalks. They too want to get from point A to point B—but what matters along the way is strikingly different from the needs of those in vehicles. For instance, when you’re walking in a busy area, you care about crosswalks. And if you’re using a wheelchair or mobility scooter, or pushing a stroller, you need to know where to find cuts in the curb.

Collecting and conveying such details was a key motivation behind the detailed new maps that Google began rolling out last year, starting with London, New York, San Francisco, and Tokyo. Now it’s announcing plans to reach a total of 50 locations—including Berlin, São Paulo, Seattle, and Singapore—by the end of this year.

Oren Naim—a 14-year Google veteran and, like Dicker, a Google Maps director of product—acknowledges that expanding coverage to even 50 cities may sound like a smallish whoop. “The thing is, technically speaking, it’s incredibly hard to make these kinds of high-fidelity maps,” he stresses. The techniques that Google used to get to 50 cities can scale way up. And as with the hard-braking avoidance feature, they were possible only because of AI.

advertisement

To suss out additional detail for its maps, Google trained AI to spot road elements in aerial, satellite, and Street View imagery. [Image: courtesy of Google]
To find the pedestrian-friendly elements it wanted to add to its maps, Google turned to photographic imagery, including satellite and aerial photos as well as its own Street View. At first, human beings eyeballed the images for the items in question and recreated them inside Google Maps. But “there’s no way that you can do that for the entire world,” says Naim. “And so what we needed to do is to find a way to teach a machine essentially to draw these maps for us.”

In recent years, tech companies such as Google have made enormous strides in using machine learning to train computers to identify elements in images at scale. But the items the Maps team needed to pinpoint presented some special challenges. A cat, for instance, is always a cat; something like a crosswalk, however, can vary wildly depending on where you are.

“For us, crosswalks have this white, black, white, black kind of pattern,” says Naim. But “if you go to London, crosswalks are actually these parallel dots. Around the world, you’ll see these kinds of road details change, not even just at a country level but at a state and sometimes even a city level.” Then again, as Google trained its models, it also discovered consistency in some surprising places—Atlanta and Ho Chi Minh City, Vietnam, for instance, had sidewalks and traffic lights that resembled each other. Who knew?

advertisement

London’s dotted crosswalks are a striking departure from their striped U.S. counterparts. [Photo: courtesy of Google]
Another aspect of the job that complicated matters: The imagery Google was working with wasn’t designed for the purposes the company had in mind. For example, Google was trying to use aerial photos to spot paths in parks. But parks have trees, and trees are tall, so the paths could be tough to spot. The solution entailed “fusing together different sources of imagery to create a model that shows you the same feature from multiple angles,” explains Naim.

In the end, humans still had to help the AI algorithms correctly deduce what they were seeing. But the progress they made will pay dividends as Google strives to bring the new detail to hundreds—and then thousands—of cities. The important thing here is that when you try to fix it, you fix it not just by cleaning this instance, but you learn something new about the world and you use that knowledge for the next city,” says Naim. “And so they become better and better over time.”

The hard-braking detection and newly detailed maps are among Google Maps’ most ambitious AI-powered additions, but they’re only two of several new Maps features being unveiled at I/O. (Another one aims to be smarter about the businesses that get called out in a map—breakfast joints in the morning, for instance, and dinner spots at night.) All told, Google believes that it’s on track to deploy over 100 AI-enabled enhancements to Maps in 2021. We’re increasingly trying to build new kinds of data about what’s happening in the world into all of our experiences,” says Dicker. Which means that it might be more practical to keep track of changes to Maps that don’t involve AI.

About the author

Harry McCracken is the technology editor for Fast Company, based in San Francisco. In past lives, he was editor at large for Time magazine, founder and editor of Technologizer, and editor of PC World.

More