A warning about sensors and surveillance

I often tell people that the reason I’m so excited about the IoT is because the combination of sensors, ubiquitous wireless, and cheap computing can be used to make the invisible visible. The benefits of using granular air quality monitoring to hold polluters accountable or leveraging an individual’s health data to help prevent or cure disease is pretty heady stuff.

But it’s also a two-way street, as all of those sensors can also make the things we’d prefer to keep invisible visible. Maybe it’s that second helping of ice cream from the night before or a sexual fetish that you’d rather your activity tracker or smartphone not have access to. Everyone has things they’d rather keep private.

When it comes to privacy, most people are focused on a limited number of sensors — typically cameras and microphones. For people with security cameras in their homes the fear of having pictures of them naked show up on the web is legitimate. So is the fear of having Alexa accidentally listen in on a conversation and a real person overhearing it.

Computers can use data from a surprising number of sensors to recreate insights that most people aren’t aware of. For example, I worry about the level of data radar sensors can gather about a person. Radar is excellent when it comes to detecting movement and can do so with millimeter-level accuracy. Thanks to machine learning, teaching a radar what various movements look like is easier than ever.

Radar isn’t deployed in many private applications yet, but accelerometers are. This paper from three German researchers at the Berlin Institute of Technology is a wake-up call for the detection capabilities of accelerometers. The paper was published in 2019 but got a new life on Twitter thanks to one of the researchers discussing it in depth this week.

In the paper, the authors reviewed and cataloged a broad swath of papers that shows how researchers could use accelerometer data in activity trackers or phones to infer quite a bit about people, ranging from their sobriety to their location and gender.

Here’s a quick list of some of the paper’s more notable findings:

• Activity data: This includes the typical inferences made to track whether someone is running or biking; it can also be used to detect if someone is driving — even if they’re driving drunk. It can track if someone is lifting an object and how heavy that object is. It can also detect if someone is smoking, which is something you might want to consider if you’re a smoker and your employer both provides fitness trackers and requires employees to avoid cigarettes.
• Identity: Using device fingerprinting of the accelerometer itself is one way sensor data can track an individual, but accelerometers can also be used to detect differences in gait, age, and gender. Researchers have even used accelerometer data to detect the height of a person’s shoe heel.
• Sound: Accelerometers can detect sound, including any hot words researchers elected to train on and look for, the emotions in a person’s voice, and their gender.
• Location: Using a smartphone’s accelerometer, researchers can calculate a vehicle’s motion trajectory and turns, and then map that data against a route to figure out the location of the vehicle with what the paper’s authors describe as being “comparable to the typical accuracy for handheld global positioning systems.”
• Keystrokes: Researchers have used accelerometer data to figure out the PIN code or password settings on a particular device, and even entire text messages typed on a phone’s touchscreen.

The only thing stopping companies or governments from trying to use this type of data is the challenge of building algorithms that can accurately detect these things and the question of why a company might want to know this level of information. However, as the cost of building such algorithms becomes easier and cheaper, it’s possible that what was once invisible will be visible.

And that visibility may not work in the consumer’s favor.