Pixel 3 and on-device AI: Putting superpowers in your pocket
Last week we announced Pixel 3 and Pixel 3XL, our latest smartphones that combine the best of Google’s AI, software, and hardware to deliver radically helpful experiences. AI is a key ingredient in Pixel that unlocks new, useful capabilities, dramatically changing how we interact with our phones and the world around us.
But what exactly is AI?
Artificial intelligence (AI) is a fancy term for all the technology that lets our devices learn by example and act a bit smarter, from understanding written or spoken language to recognizing people and objects in images. AI is built by “training” machine learning models—a computer learns patterns from lots of example data, and uses these patterns to generate predictions. We’ve built one of the most secure and robust cloud infrastructures for processing this data to make our products smarter. Today, AI helps with everything from filtering spam emails in Gmail to getting answers on Google Search.
Machine learned models in the cloud are a secure way to make Google products smarter over time.
Bringing the best AI experiences to Pixel 3 involved some re-thinking from the ground up. Our phones are powerful computers with multiple sensors which enable new helpful and secure experiences when data is processed on your device. These AI-powered features can work offline and don’t require a network connection. And they can keep data on device, private to you. With Pixel 3, we complement our traditional approach to AI, where machine learning and data processing is done in the cloud, with reliable, accessible AI on device, when you’re on the go.
The most powerful machine learning models can now run directly on your Pixel to power fast experiences which work even when you’re offline.
Benefits of on-device AI
We’ve been working to miniaturize AI models to bring the power of machine learning and computing in the cloud directly to your Pixel. With on-device AI, new kinds of experiences become possible—that are lightning fast, are more battery efficient, and keep data on your device. We piloted this technology last year with Now Playing, bringing automatic music recognition to Pixel 2. This year, your Phone app and camera both use on-device AI to give you new superpowers, allowing you to interact more seamlessly with the world around you.
On-device AI works without having to go back to a server and consumes less of your battery life.
Take Call Screen, a new feature in the Phone app, initially launching in English in the U.S., where the Google Assistant helps you screen calls, including from unknown or unrecognized numbers. Anytime you receive an incoming call, just tap the “Screen Call” button and on-device speech recognition is used to transcribe the conversation from the caller (who is calling? why they are calling?) so you can decide whether to pick up, hang up, or mark as spam and block. Because everything happens on your device, neither the audio nor transcript from a screened call is sent to anyone other than you.
Call Screen uses on-device speech recognition to transcribe the caller’s responses in real time, without sending audio or transcripts off your phone.
This year’s Pixel camera helps you capture great moments and do more with what you see by building on-device AI right into your viewfinder. New low-power vision models can recognize facial expressions, objects, and text without having to send images off your device. Photobooth Mode is powered by an image scoring model that analyzes facial expressions and photo quality in real time. This will automatically capture smiles and funny faces so you can take selfies without having to reach for the shutter button. Top Shot uses the same kind of image analysis to suggest great, candid moments from a motion photo—recommending alternative shots in HDR+.
Playground creates an intelligent AR experience by using AI models to recommend Playmoji, stickers, and captions so that you can express yourself based on the scene you’re in. And without having to take a photo at all, image recognition lets you act on info from the world around you—surfacing Google Lens suggestions to call phone numbers or show website addresses—right from your camera.
Pixel 3 is just the beginning. We want to empower people with new AI-driven abilities. With our advances in on-device AI, we can develop new, helpful experiences that run right on your phone and are fast, efficient, and private to you.
