In the fall of 2010, Google sprung a surprise on the commuting world: It had developed a fleet of seven "self-piloting" Toyota Prius Hybrids and they had already logged more than 140,000 miles (combined) in the real world, mostly on Highway 1 between San Francisco and Los Angeles. Each vehicle was powered by unproven artificial intelligence software and outfitted with a sophisticated yet awkward-looking GPS array, detailed mapping software, and optical radar and laser sensors. The car could stay on track, know where it was going, and "see" what was happening around it. And Google proved it alongside blissfully ignorant commuters and other potential plaintiffs. It caused a huge uproar. But not like you think. People weren’t pissed because they’d been unwittingly sharing the roadways with driverless cars, they were pissed because Google wasn’t sharing the driverless cars with them, Google says.
"One moment, the idea of a self-driving automobile was pure fantasy, something out of a science fiction film," says Andrew Chatham, senior staff engineer and off-board software lead for Google’s self-driving car program. "The next, it was a reality, and people immediately wanted to know more about it and when they could buy one." But it wasn't really Google's geeky, tricked out Prius that they wanted to buy. They wanted their piece of the driverless car future where the daily grind of gridlock disappears along with the technology that handles it. Then they could finally spend their commutes catching up on work, the day’s headlines, or even sleep. Such fantastic self-driving cars would potentially save 30,000 lives per year and prevent nearly 2.2 million driving-related injuries, too.
Google's autonomous vehicle has gone on to log 300,000 accident-free miles, mostly on the highways of California and Nevada. Audi, BMW, GM, Nissan, Toyota, and Volvo all have announced plans to "unveil" an autonomous car by 2020. Google says to watch for its public debut of its prototype in 2016. Still, a truly autonomous vehicle, one capable of dealing with any real-world situation, won't hit showrooms coast-to-coast for years after that
The issue isn't the technology. It’s us. The winner of the race to roll out a mass-produced AV is the one who can finally convince us to take our hands off of the wheel.
"We have the technology, and have had it for years," says Google's Chatham. "It’s just a matter of refining it to a point that it is reliable and can be trusted 100% of the time."
Car manufacturers are in full pitch mode now, eagerly enticing customers to give up more control of their cars. In 2011, newly minted GM CEO Mary Barra (then a Senior Vice President of Global Product Development) talked up the company's connected systems to an audience at the dedication of a GM China technical center, "It is this kind of inter-connectivity that will one day enable autonomous driving," she said, "and the promise of an accident-free and congestion-free future."
They tout how AVs could cause fuel consumption to shrink by more than 2.4 billion gallons per year due in part to improved traffic flow. If AVs were shared in Zipcar- or Lyft-like arrangements, the number of total cars in operation could be reduced by up to 90%. The elderly, disabled, and visually impaired would find themselves suddenly mobile.
There's plenty of money to be made in AVs, too. Experts say that some $2 trillion a year in car-related revenue and even more market cap would be up for grabs in the U.S. alone for those who dabble in vehicle automation systems. The technology business behind autonomy could dwarf Google’s current $30 billion-plus a year search-based business.
Today, three years after its big announcement, Google is still considered the leader in AV research and development. But nearly every major auto maker has jumped in the fray—Audi, BMW, Ford, GM, Lexus, Mercedes-Benz, Nissan, and Volvo (it put 100 driverless cars on the road in Sweden) to name a few. Many of the top tier 1 automotive suppliers (such as Continental and Visteon) have lept into the space, as have some top consumer electronics outfits (Cisco and Intel) and tech companies. Google owns 310 patents and 153 published US patent applications related to various levels of vehicle autonomy. Samsung, LG, Sony, Tesla, Nokia, Apple, and even BlackBerry have hundreds of patents relating to cars of the future, according to patent analysts Envision IP.
"I believe that self-driving vehicles have much to offer society," says Chris Urmson, the director of Google's driverless car program. "It's a matter of getting the technology reliable and then delivering a compelling product that makes people's lives better. If we can do the latter, then mass acceptance will follow; until then, the outcome is open to debate."
Plenty has been written about the technological milestones of various driverless cars—Google's car, in particular. This story is about how it feels to ride in or drive all these cars. In the course of reporting this story I rode in autonomous or semi-autonomous cars from Infinity, Mercedes, GM (Buick, Cadillac) and Toyota (in Japan). (See the "What It Feels Like" tales with the video clips in this story.) Then I posed various versions of one essential question to experts, technologists, and automakers involved with the cars I experienced: How will you convince us to take our hands off the wheel?
New drivers, immersed in computing technology since birth, might be more trusting than older drivers, and young people are reportedly much less interested in the act of driving than those of us of a certain age were. And it might be the case that aging baby boomers, raised with a love of cars and independence but faced with diminishing driving abilities (health issues, vision loss, etc.) would embrace driverless cars as the best of the options available to them. But for the most part, we like to be in control. Which is why the idea of driverless cars has, arguably, taken a century to catch on.
"Fact is that carmakers have been replacing conventional bits and pieces with automated systems or so called driver aids since the early 1900s," says Roger Lanctot, associate director at Strategy Analytics, a Boston-based research and consulting firm that specializes in emerging technology and mobile and wireless digital consumer and automotive electronics companies. "Take something as easily overlooked as your car’s starter, for instance." Or an automatic transmission, anti-lock brakes, or cruise control. "They are a natural part of the driving experience, as all automation should be," says Lanctot.
Other vehicle systems that have been given the automated or semi-automated treatment include the transmission, accelerator (cruise control), brakes (ABS and traction control), and suspension (stability control), to name just a few; all are essential or near essential driver aids designed to lessen drivers' burdens without intruding on the driving experience.
Over the past few years, the progression of car automation has accelerated to include systems that perform much higher-level driving tasks that are normally the sole responsibility of the driver, such as monitoring a car’s blind-spots, properly maintaining speed and distance while following another automobile, or engaging the brakes when a collision seems imminent. Consequently, today’s automated more directly impact the driving experience—they're intrusive—which can be very hard for a person to trust and surrender to, especially when it can mean the difference between life and death.
We all think we’re excellent drivers, and for the makers of self-driving cars, that's a problem.
"Approximately 80% of people we polled boasted about their prowess behind the wheel," says Dr. Bryan Reimer, research scientist in the MIT AgeLab and the associate director of The New England University Transportation Center at MIT. "They couldn’t be more wrong." Dr. Reimer leads a multidisciplinary team of researchers and students focused on understanding how drivers respond to the increasing complexity of the operating environment. They're finding solutions to the next generation of human challenges associated with distracted driving, automation, and other in-vehicle technologies. "For this reason alone it will be hard for people to accept how technology is going to change the relationship between machine and operator."
Call it the ego factor, but automakers are slowly integrating driver aids into current model vehicles at a pace consumers will be most likely to tolerate—if they recognize them as driver aids at all. "Too much automation, too soon will frighten the average car buyer," says Jeremy Salinger, General Motors research and development manager for Super Cruise, the automakers upcoming semi-autonomous driving system. "It has to be done right or potential customers will be scared off, making it even harder for us to transition from driver to driverless down the road." And if automakers and other companies aren’t sure they can sell the idea, they won’t invest in the technology. So some go half way to autonomy with varying results.
Take the all-new 2014 Infiniti Q50 sport sedan. During a recent test drive, we were able to go about 35 miles at 75 mph on a somewhat hilly, not-so-straight but not-so-curvy section of I-95 North, just above Boston, without ever touching the steering wheel, accelerator, or brake. It worked. But it often hugged the paint of the lane divider too closely for my comfort—the weirdness of it all made the experience barely less nerve-racking than navigating rush-hour traffic.
Plus, when you do manually steer the car, you don't feel the bumps and bruises in the road through the steering wheel. "Unlike in a conventional car, there is no mechanical connection between the steering wheel and the front tires in this setup," says Bert Brooks, senior manager of product planning for Infiniti. "Input from the driver is relayed to a series of control units. They control the front wheels." The benefits of such a system are that it allows independent control of the Q50's tire angle and steering inputs and is supposed to translate a driver’s intentions to the wheels faster than a mechanical system.
Mercedes-Benz is offering a similar driver aid on several models. Unfortunately, neither system is perfect. Yes, they are designed to give a motorist the ability of hands-free driving when the system determines it is safe to do so. No, they aren’t meant to transform the vehicle into a driverless drone. They can only navigate around a certain angle before the system alerts and diverts back to driver control. Unfortunately, that turn angle isn't always consistent.
General Motors also recently demonstrated its Super Cruise semi-automated lane keeping technology at the company’s Milford Proving Grounds in Michigan. Super Cruise is capable of hands-off lane following, braking and speed control under certain driving conditions. It is designed to ease the driver’s workload on freeways only, in bumper-to-bumper traffic and on long road trips; however, the driver’s attention is still required.
While the goal of Super Cruise is sounds similar to Infiniti’s Lane Control, it is a much more refined system. It considers more variables (sensor data), such as GPS and detailed mapping software, when accessing the driving conditions and determining how the car should react in any one given situation. Thus, it is more precise and reliable.
"Super Cruise is designed to give the driver the ability of hands-free driving when the system determines it is safe to do so," said John Capp, GM director of Global Active Safety Electronics and Innovation. "Before we introduce this capability on a production vehicle we must put the system through rigorous testing and technology refinement."
So why not combine all of these technologies for something closely resembling a truly autonomous car? Because, "This is where the real problems start," says Google’s Chatham.
First, the data collected by the car’s network of sensors, which could be anything from road and weather conditions, to pedestrian activity and what other cars are doing in your vicinity; then it has to be compiled and translated into a language that a computer can easily understand; and then the data must be interpreted just like a human operator would. Finally, after assessing the situation, the car’s artificial intelligence must formulate a response and instruct every system what to do—brake, turn, accelerate, swerve at the same time—in order to, say, miss a child in the road.
"Making this happen consistently and reliably is a problem with a very long tail," says Chatham.
Existing driver aids like Adaptive Cruise Control, which use radar to measure the distance to the car in front and controls the accelerator and brake to maintain a safe distance; or a Lane Control System, which uses a camera in the rearview mirror to monitor lane markings and vibrates the steering wheel, or gently moves it, if the car drifts too close to the paint left or right, employ what Lanctot calls "first-order intelligence." Each system can make a single decision using one particular set of variables. But they are only making one decision.
"The driving experience is much more diverse than that," Chatham says. The artificial intelligence that runs the car needs to make multiple, almost sentient decisions based on multiple sets of data, a process called "second-order intelligence."
"We are currently mapping out the all the variables that can occur under any driving situation so that our decision-making algorithms are more robust, more reliable. We have come a long way in the last three years, but haven’t gotten 100% efficacy yet."
The National Highway Traffic Safety Association classifies four levels of autonomy, ranging from ABS and Adaptive Cruise Control, to full drone-like driverless vehicles. In clinical terms, "developers are somewhere in the Level 2 to Level 3 range," says Strategy Analytics Lanctot. Vehicles at this level enable the driver to cede full control of all safety-critical functions under certain traffic or environmental conditions and in those conditions to rely heavily on the vehicle to monitor for changes in those conditions requiring transition back to driver control. That is, the vehicular equivalent of Google’s autonomous cars, allowing users to cede almost all control to some sort of artificial intelligence.
When will we transition to a Level 4, drone-like, "Look ma, no hands" fully automated vehicle? If Chatham is right, that will happen over the next few years. Unfortunately, there is a lot of work to be done refining driver algorithms, so that all maneuvers are correctly executed 100% of the time.
Then there's the question of manufacturing—and what will be manufactured by whom. It doesn’t make sense for Google to get into the autos game. They would make more money as a developer of AI software (selling their secrets once perfected) for more developed automakers, than to actually turn around and build them.
Consequently, the first decade of AVs would probably be a mixed bag of semi-autonomous vehicles. Fully self-driving cars are more likely in 20 years. The fleet of cars on the road turns over roughly every 10 to 15 years, so even if driverless cars were in production today it would be many years before they dominated our highways and started delivering the promised benefits.
In the meantime, you’re going to see carmaker rolling out the technology in small doses, getting people used to using driver aids and having less responsibility behind the wheel.
"AVs are inevitable. It’s only a matter of time," says Google’s Chatham. "They are the most logical next step and will have socio-economic and other effects on our society—some good and some bad—quite possibly changing our whole way of life. People will have to accept it."