Google assembles all the parts of a truly smart home

Ever since Google Now was introduced in 2012, I’ve said, “Context is king.” That’s because it proved that if a system knows who you are, where you are, and what you’re doing, it can contextually deliver information or actions that are hyper-relevant to you.

But while hyper-relevant information and services are useful when you’re out and about, they’re even more useful when you’re at home. And now Google has the pieces to deliver them in the smart home.

Some of those pieces Google has built up over years — namely, the treasure trove of personal data users provide to it when using its services. Many people rightly focus on how Google uses that information to provide you with contextual ads. But if you use Google Maps, it also knows where you are, where you’ve been, and where you might be going.

Search on your computer for a hardware store, for example, and Maps can surface the location on your phone when you leave the house. And if you’ve configured it with your home and work address, Google matches time and location with those destinations. It’s why you might see a notification telling you to leave for work earlier than usual due to traffic.

Context in the home is a challenge

The type of contextual information that Google fleshes out on you is more difficult to gather when you’re at home, as most of us are these days. That’s because, in the home, Google needs different information about you. And it’s not information that you might type into a search box or ask your Google Assistant.

A smart home needs to know who is home, what room they’re in, and how they may like their environment to be, such as in terms of lighting and temperature. That contextual information comes from sensors, cameras, microphones, and the like. It’s why back in May Google added a feature to its Nest Aware subscription plan that alerts you when your Nest products hear certain sounds. Broken glass indicating a break-in and sirens from smoke alarms to know if there’s an emergency situation were the first two contextual alerts offered. Those are relatively simple audio events to listen for, but Google has AI and ML smarts that could listen for and interpret less clear-cut situations, too.

Every smart device is a data endpoint

Even without a camera in your kitchen, a nearby Nest audio product would be able to hear the sound of running water, know which direction the sound is coming from, and recognize that it’s the kitchen sink. In other words, even without video proof Google can detect that someone is present in the kitchen.

Let’s take things a step further. Suppose you have a connected “Works with Google” toothbrush that Google knows you use every morning just prior to leaving for work. By taking that context and combining it with other information it already has, Google could offer to remotely start your car as you brush when the temperature outside is below freezing, because why sit with clean but chattering teeth for the first five minutes of your commute?

That “Works with Google” program is a key element here, and not just because it may allow voice control for devices. Each of these supported products is also an endpoint for Google to gather more contextual information about us and our homes.

Don’t forget about Google’s “Voice Match” feature and its move towards local processing. Currently, Google matches contextual information with the person asking about their calendar events and such. It knows who is asking the question, so it can potentially determine who is at home or even who is in what room simply by hearing that person’s voice.

Yes, that sounds invasive. However, Google is making a push for localized actions on its smart speakers so that the captured audio and other data stays in the home instead of getting uploaded to its servers. This could remove some of the privacy invasion feelings people may have.

Presence detection is camera-less and invisible

Google has also been making advances in detecting people in front of its smart devices. Did you know that the Nest Hub, Nest Hub Max, Nest Mini, and Nest Wifi all permit ultrasonic pulses to detect when someone is nearby?

Note that the latter two devices don’t have cameras. Yet Google is able to detect presence in the smart home with an alternative solution. It can do the same with Nest thermostats as well, with a traditional IR proximity sensor.

And just this week, it was reported that Google is testing a project called “Blue Steel” on the Nest Hub Max. Simply by standing in front of the smart speaker, a person can ask Google questions or issue smart home commands without saying the “OK Google” hotword. Since “Blue Steel” is still an internal project, it’s not yet clear if Google is using ultrasonic pulses for person detection. One way or another, a camera-less approach to context in the smart home is important because it eliminates something many people, including my family, don’t want — cameras in every room of the house.

The smart home has come far in two short years

Whether a smart home uses cameras, microphones, or other sensors, it all brings me back to the 2019 Consumer Electronics Show. After walking miles and miles through a sea of connected gadgets, I decided that the smart home still wasn’t smart.

What would it take, I wondered? “At some point, we need devices that require less user interaction from a voice or programming standpoint to make them smart,” I wrote back then. “Instead, by algorithmically learning our personal preferences and schedules, a smart home with predictive intelligence should optionally suggest what to turn on, watch, or listen to as well as when, where and in the best lighting/temperature climate for a truly personalized and smart home experience.”

We’re not quite there yet, but by advancing and maturing its products, combining them with AI, ML, and our data profiles, Google has much of what it needs to offer an intelligent, contextual smart home experience.

Join Kevin, Stacey, and people from companies including Wyze, Ikea, and more on Tuesday, October 27 for Level Up the Smart Home. Register today.