Let’s ask a teen about Amazon’s really odd idea for Alexa

Sometimes you read something so weird you do a double take, and subsequently find yourself searching for the original source or more information because there’s simply no way that someone really said whatever you just read. I had such a moment this week after seeing someone tweet about Amazon wanting to create a deep fake that allows Alexa to channel a dead family member.

I’m sorry, what?

The reporting on which the tweet was based was done at the Amazon Re:MARS event that was held on Wednesday, where Amazon’s senior vice president and head scientist for Alexa, Rohit Prasad, showcased how well Amazon can mimic speech based on just a one-minute sample of someone’s voice. As TechCrunch reports:

In the scenario presented at the event, the voice of a deceased loved one (a grandmother, in this case), is used to read a grandson a bedtime story. Prasad notes that, using the new technology, the company is able to accomplish some very impressive audio output using just one minute of speech.

TechCrunch labeled this as a “potential new feature.” So I reached out to Amazon to get a bit more information. But I’m not sure if Amazon’s spokespeople can really answer the biggest question, which is: Why? Why would people want to hear a deep fake of a dead loved one read them a story? Or give them the weather?

While I too would love to hear the voice of a deceased loved one read me a story, what I really mean is I’d love for them to be alive again. A faked version of a loved one drops immediately into the uncanny valley, and lodges there like the foundation of a new dystopia.

But maybe I’m just old and jaded. So I decided to ask my teenager for their thoughts on Amazon’s idea. Alas, their first question made clear that my cynicism had found a home in my offspring. “Would Amazon make this so they could sell it to people because they miss their grandparent?” they asked. “Is this about money?”

I explained that it was a way to showcase the advances of AI when it comes to being able to replicate people’s voices. Then I asked if they might like to hear my voice if I was dead or merely away on travel. “But I’d know it wasn’t you,” they said. “This robot would be using your voice.”

In other words, they weren’t having any of it. Now, my child isn’t a representative sample of the population by any stretch. But the fact that a 15-year-old was so quickly able to pinpoint how creepy this example is and express their horror within just moments of hearing about it had me wondering what, exactly, the folks at Amazon were thinking.

Is this a simple case of the engineering department getting excited about technology and trying to explain why it’s so neat in a way that is, for many normal people, tone deaf? Or overblown reporting that took a careless statement and blew it up into a big thing? Or is Amazon testing the waters to assess the general readiness for a technology that seems creepy, but might actually entice some people? (I think of Amazon announcing the flying Ring Always Home Cam two years ago as an example of this.)

I often read the news and find myself picturing how new technology can be used against people, or to extract revenue from people while providing them with little actual value. For example, I still have deep questions about the effects of paying for hardware as a service and how it impacts consumers’ ability to build value through their assets. And when I look at efforts to gather more and more personal health data in one place by offering new “features,” I tend to think technologists might be selling us digital snake oil.

But this particular demonstration struck me as so tone deaf that I’m left wondering what the positives would be for anyone, even Amazon. If it wants to sell us more celebrity voices on Alexa, invoking a dead grandmother and a bedtime story seems like a terrible way to do it.