Google’s AI Chief On Teaching Computers To Learn–And The Challenges Ahead

When it comes to AI technologies such as machine learning, Google’s aspirations are too big for it to accomplish them all itself.

Google’s AI Chief On Teaching Computers To Learn–And The Challenges Ahead

At last year’s Google I/O keynote, CEO Sundar Pichai laid out the company’s vision for a new era of AI-first computing. AI mattered at least as much at this year’s keynote on Wednesday. But instead of being a vision, it was more of a fact of life. That’s because it was everywhere–in Android, Google Photos, the Google Home speaker, the Google Assistant (which will add a machine-vision feature called Google Lens), and more.

John Giannandrea

After the keynote, I caught up with Google senior VP of engineering John Giannandrea–who, though he didn’t appear onstage, is deeply involved in all of the above efforts and others as the company’s lead for AI. “Last year, we talked about becoming an AI-first company and people weren’t entirely sure what we meant,” he told me. With this year’s announcements, it’s not only understandable but tangible.

The news from I/O that Giannandrea and I discussed was as wide-ranging as the keynote itself:

Intelligence Everywhere

At I/O, Google announced–which is maybe less of an actual thing than a statement (and accompanying website) designed to remind the world of the company’s ambitious and far-flung efforts in AI. Giannandrea calls it “an umbrella brand” that shows off Google’s work in hopes of inspiring others to build upon it. “We’re saying, ‘Come use this amazing stuff, see what you can do,” he explains.

It’s not just about Google’s own apps, services, and gadgets. The AI techniques created by Giannandrea’s team may make their way into Google products such as Lens; be turned into on-demand Google Cloud offerings that can power other companies’ software and services; be leveraged in research efforts such as the company’s work to identify breast-cancer tumors; and be open-sourced so that anyone can use and tweak them for purposes beyond Google’s own ambitions.

“We see our job as evangelizing this new shift in computing,” Giannandrea says.

Giving Google A Lens

I knew that Google Lens–which will be part of both the Assistant and Google Photos–scratches a long-standing Google itch because I remember Google Googles, an app which launched way, way back in 2010. Conceptually, they’re the same thing: You aim your phone’s camera at something in the real world, and AI understands what it’s seeing and does something useful for you. It’s just that some of the things Lens will do–such as reading a Wi-Fi password off the sticker on the back of a Wi-Fi router and then logging you in–would have sounded like fantasy in 2010. (Heck, that sounds a tad fanciful even in 2017.)


Giannandrea says that Lens isn’t the result of any particular breakthrough, but rather slow and steady process in machine vision–and, particularly, the fact that the technology is getting better at avoiding false-positive and false-negative identifications. “Computers have just been getting better at seeing,” he says.

A Google TPU
A Google TPU

AI (Not) In The Cloud

If you think of Google performing AI, you’re likely to envision a data center full of servers crunching numbers in the cloud. On that front, the I/O keynote included the announcement of a second-generation version of the ultra-powerful machine learning-optimized hardware it calls a Cloud TPU. (“It’s exciting to have 180 teraflops of hardware devoted to learning,” Giannandrea cheerfully says.) But Google is also at work on AI-infused technologies that run directly on mobile devices, an approach that has gotten more effective as it’s learned how to perform AI more efficiently and gadgets have gotten more powerful GPUs, the chips that are suited to the high-speed calculations required by AI.

TensorFlow Lite, a new offshoot of Google’s open-source TensorFlow software for creating machine-learning applications, runs directly on Android devices, enabling features such as new features in Android O that are smart enough, for instance, to notice that the text you’re trying to highlight is an address. It wouldn’t work nearly as smoothly if it had to talk to a server across an internet connection.

“You want to do machine learning on the device as much as you possibly can,” Giannandrea says. “It’s lower latency, it’s closer to the user, it’s distributed.”

Getting It Right, Again And Again

Google Home recently gained the ability to distinguish between up to six different people’s voices–a key upgrade because it unlocks the potential for the Google Assistant service to know who’s talking and respond with more customized information, such as details about travel plans drawn from a specific person’s Gmail, for instance. I knew that telling voices apart was an impressive feat because my colleague Jared Newman wrote an entire story in March about how hard it was.

“We were pretty proud to get that out the door,” Giannandrea told me. He added that the challenge, as with many uses of machine learning, is not to prove it can work in a lab environment, but to make it so rock-solid reliable that it’s ready to use in the real world and is more likely to impress people rather than tick them off. “It’s all precision recall,” he says. “When it doesn’t work, it can be embarassing.”


Matching People With Jobs

Pichai concluded the I/O keynote by previewing Google for Jobs, an upcoming career search engine that uses machine learning to understand job listings–a new approach that is valuable, Giannandrea says, even though looking for a job has been a largely digital activity for years. “They don’t do a very good job of classifying the jobs,” Giannandrea says. “It’s not just that I’m looking for part-time work within five miles of my house–I’m looking for an accounting job that involves bookkeeping.”

I wondered if the fact that Google Jobs was the keynote’s final bit of news was an oblique acknowledgement of concerns that AI will eliminate jobs on an unprecedented level in the coming years. Declaring himself “pretty optimistic” about AI’s potential to make the world a better place, Giannandrea said that countering dystopian fears wasn’t the intent. “It’s really a reflection of the fact we want to make tools that are useful for everybody,” he says. “There’s a perception that technology isn’t as relevant for every demographic. We strongly believe technology should help everybody, no matter what they’re trying to do.”

Challenges To Come

Google has worked its AI into so many aspects of so many of its ventures over the past year that I asked Giannandrea whether he thought it could keep up the pace in the years to come. “I actually think we’re speeding up the rate of acceleration,” he told me. “But it’s obvious there are problems we don’t have solutions for.” For instance, as uncannily effective as Google search can be, and despite the power of its knowledge graph, computers don’t truly understand what humans write. Google would like to teach them to “read anything and summarize anything,” Giannandrea says.

Another big challenge is to create machine-learning technologies that can understand facts about the world with less training. Giannandrea notes that his 4-year-old daughter saw an old-school penny-farthing bicycle, knew it was a bike, and–once he’d told her about it–was set to identify penny-farthings from then on. Today’s computers would need a lot more data to learn the same thing.

“We’d have to show them 100,000 penny-farthings and tell them it’s a bike,” Giannandrea says. Then–looking on the bright side–he adds that “once they’d seen 100,000, they’d probably be better at identifying them than humans are.”


About the author

Harry McCracken is the technology editor for Fast Company, based in San Francisco. In past lives, he was editor at large for Time magazine, founder and editor of Technologizer, and editor of PC World.