Yesterday, at the first day of its annual developer conference I/O, Google cited a staggering number: There are now 2 billion active monthly Android devices, with 1 million more devices coming online every two months.
That’s a hell of a lot of smartphones and tablets–and with Android on TVs, cars, watches, and laptops, it’s only going to grow. To power this growing ecosystem of devices, Google released a beta for developers of its latest operating system, Android O. In addition to some performance improvements, O will put some small, clever UI tweaks on billions of devices. Here are a few we saw today at I/O.
Using two apps at once–say, Skyping while looking at your calendar–is usually a total pain on mobile. But a new O feature called “Picture in Picture” shrinks the video interface so you can look at another app simultaneously. Take watching a Youtube video. Currently, if you close Youtube to answer a text, you can’t see the video playing anymore. Picture in Picture moves the video to the bottom righthand corner where it continues to play, letting you multitask much more efficiently. You can watch Netflix while Googling a character, or maintain your video call with a friend as you schedule an event.
The new operating system also shifts how notifications happen within Android. O’s new design takes some cues from Apple, adding a small dot to app icons to indicate a new notification. Also somewhat similarly to iOS, a long-press on the icon will pull up a small window showing the notification without launching the app. The change doesn’t make new work for developers–the system pulls the color of the notification dot from the app icon automatically.
Smart Copy And Paste
Another pain point in Android today? Highlighting text to copy and paste. While previously you had to long-press or double tap to begin a selection, and then drag the UI handles on either end of the word to increase or decrease the selection, Google has put its machine learning to the work improving that process in O.
Google’s user testing revealed that the most highlighted text on mobile phones is phone numbers, followed by people’s names, businesses, and addresses. The new smart text selection feature in Android O uses AI to recognize which of these you’re trying to select. One tap will select the entire phone number, name, or address, and the UI then presents context-specific options: the phone app for a number, Gmail for an email, or Google Maps for an address. All this happens on the device in real time.
Autofill For Apps
One of Google Chrome’s handier features on desktop is its autofill system, which will automatically input information like addresses, passwords, or credit cards. Now, autofill will be available across all apps on Android O, making logging into apps on a new phone easier than ever. And it syncs as well–if you logged into Twitter on Chrome via autofill, logging into Twitter on Android will be autofilled as well.
Smart Reply In Gmail
Gmail’s Inbox app and the messaging app Allo already have a handy feature called Smart Reply, where Google’s machine learning provides three context-dependent replies to each email you receive. Now, it’s coming to Gmail proper for iOS and Android. While this UI change isn’t Android specific, it will definitely make any Gmail user’s life easier–regardless of what phone or operating system you use.
— Google (@Google) May 17, 2017
AR For Everyday Decisions
One of the most exciting announcements on the first day of Google I/O was Google Lens, which uses machine learning to recognize images in the real world, on the go–making it almost like a visual search.
Google CEO Sundar Pichai demonstrated one particularly compelling application of the tool, which will live inside Google’s AI Assistant: Hold your phone up, scan the street in front of you, and Google Lens will recognize restaurants, popping up a menu in real time over the image that shows the restaurant’s phone number, menu, and reviews, just like what you’d get on Google Maps. It’s a hugely practical application of AR for everyday life.
How Is All Of This Possible, Anyway?
Many of these UI tweaks are powered by machine learning–which takes a huge amount of data to process. In order to make it all feasible on mobile and open up these types of capabilities to developers, Google announced a new version of its open-source machine learning library, TensorFlow. Called TensorFlow Lite, it’s a library designed specifically to be fast and small for mobile devices, enabling developers to use machine learning techniques in their apps. And coming later this year? A neural net API.
We’re seeing a new level of integration across all of Google’s services and products–from Android O and the Pixel phone to Google Home and Google Lens–and, more and more, machine learning is weaving them all together.