Former Googler creates Tiny ML sensor startup

What if we could take the internet out of the Internet of Things? Or at least take the internet out of some of the things? That’s the idea behind Useful Sensors, a startup created by Pete Warden, the former technical lead of the TensorFlow Micro team at Google, and previous founder of Jetpac, a deep learning technology startup acquired by Google. With Useful Sensors, Warden wants to solve the problem plaguing the Tiny ML community — namely that there are few celebrated use cases outside of automatic wake word detection.

But the promise of Tiny ML is huge. By embedding machine learning on the sensor itself engineers can design smart products that don’t need an internet connection, can save power and can boost privacy because the data stays locally on the device. Those benefits can accrue to any machine learning that takes place on the edge, such as a phone or a smart speaker, but Tiny ML is designed for constrained computers without much power or memory.

To run any sort of machine learning on the microcontroller-powered devices takes different types of algorithms, maybe different processor architectures, and a willingness to accept a little less certitude in the accuracy of the model. But the benefits are still powerful. Warden asked me to imagine a TV that might turn on and show what I was last watching, or a light that turned on to the right setting based on my preferences. His team is also building an oscillating fan that can follow a face, aiming the breeze either directly at someone or away from them depending on their preference.

Useful Sensor’s first product is a computer vision sensor (a tiny camera with a 110-degree field of view) that can detect people and distinguish between a few faces. It’s good enough for personalization of lighting or figuring out who was watching TV, but not accurate enough that Warden felt comfortable offering it in a lock or security sensitive device.  The sensor is available on the Sparkfun site for developers who want to play with it for $10.

The price, the ability to buy it in one-off or small batches on a web site, and the functionality make this incredibly accessible for everyone from DIYers to engineers who want to prototype something within a large company. Warden’s hope (and mine) is that we could see the sensor set off a wave of use cases where having an offline device recognize a person changes the way it could function. For example, what if GE or Bosch embedded something like that inside a stove faceplate so kids couldn’t turn the stove on? Or a sensor inside a mirror could be used to call up a person’s news and appointments for the day? Automotive companies could use such a sensor to determine someone’s head position or whether or not a child was left in a carseat.

The $10 person-detection sensor will use “tens of milliwatts” of power and is smaller than a quarter, says Warden. And next month, Warden says the he plans to launch another sensor that would track basic gestures, and allow for new types of interfaces for connected or unconnected objects. As one of the grandfathers of the Tiny ML community, Warden is incredibly enthusiastic and sincere about the potential for smarter sensors.

He’s also concerned about privacy and trying to keep the inevitable rise of everyday microphones and tiny cameras from becoming “creepy.” He’s hoping that the consumer electronics industry and the government will come to some sort of consensus around labeling and transparency so consumers buying devices understand exactly what sensors are on their new gadgets and how they might be used. I agree.

Ironically, by making an easy-to-use person-detection sensor smaller than a quarter that costs a mere $10, Warden is contributing to the rise of these devices in everyday objects. Our world is going to get smarter because the potential benefits are compelling, but we also should recognize that this technology also needs some guardrails so people at least are aware of what’s happening.

For the image sensor, Warden has made it difficult for anyone to access the raw image data from the sensor — they can only get the metadata about the face such as detection, identity and if the face was looking at the sensor. The sensor also is pre-programmed and developers aren’t allowed to flash or reprogram the sensor, so they can’t turn it into a tiny low-power, low-resolution spy cam. Warden has also designed the sensor so third parties could audit it, should standards bodies or governments try to create rules around privacy and smarter sensors.

Warden’s approach here is thoughtful and could help solve the lack of use cases that’s currently keeping Tiny ML more of a research project as opposed to a world-changing technology. From a business perspective Warden knows he won’t make money selling the sensor to DIY audiences, but he hopes their work inspires larger companies that make everyday objects to see what the technology can do, and eventually become customers.

I would love to see this effort pay off in the form of smarter devices that don’t necessarily need an internet connection to offer new and valuable features tied to personalization and context. Maybe the ambient home or smart home doesn’t need as many connected devices as we thought. Maybe we just need smarter sensors.