Shhh! We’re heading towards a quieter, gesture-based smart home

To say that voice has become a primary control for the smart home would be a vast understatement. In less than five years since the first Amazon Echo device was launched, smart speakers equipped with microphones have not only become mainstays in long-time smart home enthusiasts but have also helped boost connected home product sales in the US mainstream too.

So that’s it then: voice control is entrenched in our homes and will be for years to come, right?

Not necessarily, although voice commands won’t be going away any time soon. Instead, it’s more likely that another invisible interface will supplement voice in the smart home: Advanced sensors for gesture-based controls for the times when a silent interaction is preferred.

What got me thinking about this are a few of the big announcements from this week’s Mobile World Congress event. Specifically, the ones around gesture control in cars and some of the newest sensors in smartphones.

On the automotive side, we already have numerous sensors for assisted- and self-driving vehicles: radar, LiDAR, cameras and more. These, along with powerful chips and wireless connectivity empower cars to make safe driving decisions for us. But the car manufacturers aren’t stopping there. Along with voice controls — either built-in or through add-on products such as 3rd-party products — some automakers are moving ahead on gesture control.

BMW, for example, introduced its Natural Interaction platform this week at MWC, which supports either voice or gestures. And it’s not just for controlling in-car audio or opening the windows:

“With enhanced gesture recognition and the car’s high level of connectivity, the interaction space is no longer confined to the interior. For the first time, occupants will be able to interact with their direct surroundings, such as buildings or parking spaces. Even complex queries can be answered quickly and easily by pointing a finger and issuing a voice command. “What’s this building? How long is that business open? What is this restaurant called? Can I park here and what does it cost?”

Assuming BMW can pull that off by 2021, which is the current plan for Natural Interaction, that not only adds a new method of interaction but adds value to the experience as well; something that would enhance a truly smart home in the future if the tech were to navigate into the smart home.

So what do you need for a gesture-control system? Aside from the silicon and processing power to understand gestures in context, you need sensors different from the ones we use today to simply detect motion. And you may need more cameras too, which for many, is a non-starter in the house.

But we’ve seen this type of tech in homes before and not too many people seemed concerned about the privacy aspect: I’m talking about the Xbox Kinect sensor that sold 35 million units until Microsoft killed the product off in 2017. Inside the Kinect is a 640 x 480 resolution RGB camera, an IR depth sensor, and a microphone array, all of which contributed to a visual recognition and gesture-based system for specific Xbox games. Microsoft later updated the hardware in a sleeker, second model but similar sensors were used.

The basics were in the Kinect for a gesture experience but in the smart home, we’d need more advanced hardware. And I don’t just mean higher resolution cameras.

New ToF or Time of Flight sensors, like the one used in the LG G8 ThinQ smartphone, also announced this week at MWC are one component that would be useful. This sensor brings gestures to the phone, as well as the ability to unlock it by scanning the owner’s handprint:

“Without touching the phone – useful especially when driving – users can answer or end calls, take screenshots, switch between applications or adjust the volume. Controlling the LG G8ThinQ is as simple as waving a hand or pinching the air, increasing convenience for active consumers when wearing gloves or swimming.”

For gestures in the smart home, this is the type of technology that would be needed, but on a larger scale. Why? It’s one thing to use gestures a few inches away from a phone. It’s completely different if you’re 15 feet away from the TV and need to mute it.

Lighthouse actually debuted a $300 smart home camera with a 3D ToF sensor to recognize more information than a camera combined with accurate motion detection but that product is no longer available. However, with more smartphones now adopting these sensors, production costs should decrease over time, bringing new use cases and opportunities.

Even with older tech, that old Kinect sensor was pretty good at gestures; yes, we had one.

I remember that to get the Kinect’s “attention”, you would wave to it. And I could be at least 10 feet away to wake it up, “throw” a baseball, or decrease the in-game volume with a flip of the wrist. So it’s not unreasonable to expect 2019 technology to handle in-home gestures when it was crudely available in 2010. Once we have the tech, it simply becomes a question of how and where it’s used in the home in a way that can supplement voice and make the smart home smarter.

It could be a hand wave to arm or disarm an alarm system, so we’re less reliant on geofencing. Low-hanging fruit would be sound controls for smart speakers and TVs so we don’t have to shout over content for our smart home to be heard. Or maybe it’s a raise of the arm when pointing to a thermostat to boost the heat on a cold night while snuggled up with a good book.

The technology to make this happen doesn’t seem too far away or too far fetched to me. At this point, it makes more sense to start thinking of the possible use cases where you’ll want to interact with your house but still keep the noise down.