Your smart home is getting smarter, with help from Gemini

It’s been an exciting year for Google Home as we continue enhancing our smart home experiences for both users and developers. At Google I/O, we opened our platform, giving developers tools to create new and innovative experiences for the home. While we were making the smart home better, Google’s Gemini models were unlocking new ways to help people in their daily lives.

Today, we’re reinventing two of our most iconic and beloved devices with our newest Nest Learning Thermostat and the new Google TV Streamer. And we’re thrilled to unveil how we’re using Gemini models to make our devices smarter and simpler to use than ever, starting with cameras and home automation. We’re also using Gemini models to make Google Assistant much more natural and helpful on your Nest speakers and displays.

Supercharging your Nest Cameras and Google Home experiences with Gemini

Security cameras have become an important part of our daily lives in helping us keep our family safe and secure. Over time, cameras across the industry have gotten smarter — from simple motion detection, to people and package detection, to more advanced AI features like familiar face detection unique to our Nest cameras. But even with these AI improvements, it can be hard to truly understand what is happening in and around your home.

With Gemini multimodal models that can process video, images and text we’re now able to redefine camera intelligence for the industry. Over time, your Nest cameras will go from understanding a narrow set of specific things (i.e., motion, people, packages, etc.) to being able to more broadly understand what it sees and hears, and then surface what’s most important.

This means that your camera can understand that it isn’t just an “animal detected” in the video, but it will understand that “the dog is digging in the garden.” With these new AI descriptions, Gemini models are making your camera clips in the Google Home app far more useful.

You can also use the new camera activity search in the Google Home app to find exactly what you are looking for in your home. For example, “Did the kids leave their bikes in the driveway?” Google Home can search through your camera history, giving you a list of only the relevant events and a helpful summary. That simply wasn’t possible before. The sky's the limit for what you can ask, “Did the FedEx truck drive by today?” or “Did my dog go near the cookies?” This will be a monumental shift for Nest cameras and will transform how you get help in your home. And we do all of this while ensuring your data is safe and private, consistent with our principles.

We've also been working to make Google Home one of the most capable home automation platforms. Today in both our script editor and the Google Home app we support more than 100 different ways to connect and control your smart home, and our savviest smart home users love this. But with countless possibilities for what to automate, it’s hard to know which steps will make the best automation for your home.

Gemini takes the complexity out of creating automations. With the new Help me create feature in the Google Home app we will solve that challenge using Gemini models. Help me create lets you describe what you want in plain language, and Google Home will set up the automation for you.

For example, type or say “Lock the doors and turn off all the lights at bedtime” and get an automation created seamlessly. Even if you don't have an exact automation in mind, the Home app can help come up with suggestions based on what's possible with your devices. You could try “Help the kids remember to put their bikes away when they get home from school,” and get a suggestion to broadcast a reminder when someone comes home in the afternoon.

With any new technology like this, we know it will take time to get it right, so we are going to roll these features out to a limited number of Nest Aware subscribers who are in Public Preview later this year, and will expand over time.

A more helpful Google Assistant for Nest speakers and displays

Google Assistant in the home has helped millions of households get more things done. And unlike on personal devices like phones, home devices like speakers and displays need to work for every member of your household. As we’ve previously shared, quality improvements are a focus for us this year and we’re applying large language models to improve core experiences such as media playback and setting timers. And later this year we’re bringing a variety of new voices that make interacting with your devices feel more conversational.

With Gemini technology, we’re also reimagining what your home assistant can do with just your voice. This will improve Google Assistant on your Nest speakers and displays, making it more natural and helpful for everyone in your home. The upgraded Google Assistant will be able to better understand you, so you can chat more naturally, and easily go back and forth or ask follow-up questions.

You won’t have to phrase your question in a specific way — ask it in the way that feels most comfortable to you. For example, ask "Is Pluto a planet?" and then dig deeper with "Could they change their minds again?" Can’t figure out the name of that song? Just ask Google, “What’s that iconic basketball theme song.” Or Assistant can tap into Gemini’s generative capabilities to have some fun during a family discussion, like “Help me make the case that Crocs are extremely cool — and mention sport mode.”

We will initially roll out these improvements to Nest Aware subscribers in Public Preview later in the year. We’re excited and optimistic about how Gemini technology can help us build the next generation of helpful experiences in the home, and we’re committed to bringing these innovations in a way that respects you and your home.