I'll let you in on this open secret: Alexa, which is the voice assistant embedded in smart speakers sold by Amazon (Echo) as well as by some of its partners including Bose, Facebook, Sonos, Sony, and Ultimate Ears, is listening to you all the time.
Yes, even if you don't use the "wake" word "Alexa." Actually, it may happen—it does to me often—that Alexa turns on and starts speaking to you even if you didn't solicit her.
The current standard for an Alexa-powered device to obtain an Alexa certification is set to a maximum of three false alarms per day, which means that situations, where Alexa listens without the user invoking one of the wake words (Alexa, Echo or computer), are far from rare.
The limit of cloud-based voice assistants
The fact that Alexa is always listening to her surrounding is easily explained by the technology that Amazon chose to implement for its smart speakers: The Seattle-based technology giant uses cloud computing to process every spoken word captured by its smart speakers. What it means, in layman's terms, is that every word you say to Alexa is sent to Amazon's cloud service to be automatically transcribed before it can respond to your request including basic commands like "play music" or "turn on the light"—nothing is processed on the device itself because it doesn't have the necessary computing power and the intelligence on-board.
However, that's where it gets worse. Amazon has actually hired a team of people around the world to analyze snippets of conversations that Echo devices secretly record and upload—without the user's consent or knowledge—to the cloud, claiming it will improve Alexa's "customer experience". Unfortunately, the data is not really being anonymized and can be traced back to the actual owner of the smart speaker.
"Amazon Alexa is a cloud-based voice assistant so once the wake word is triggered, whether the user actually invoked it or not, the audio starts to be streamed to the cloud for Alexa speech recognition and natural language understanding to happen," told me in an exclusive interview Joseph Dureau, the CTO of Snips. "There are, thankfully, alternatives to this model. For example, Snips runs speech recognition and natural language understanding on the device itself, which means that no voice data ever leaves the device. This ensures that no private conversations are sent to the cloud, even in the case of wake word false positives."
Crowdsourcing voice samples instead of secretly eavesdropping on user conversations
In the Alexa app, Amazon says that this secret eavesdropping setting which it calls "voice recordings" can be disabled. In reality, you can only stop your recordings to be used for product development but not during "normal" operation or if Alexa listens in on conversations by "mistake". However, if you still choose to do so, Amazon warns you that "if you turn this setting off, new features may not work well for you."
"Performance is in fact only minimally improved with the collection of more data past a certain point," adds Dureau. "The volumes of data required to reach state of the art performance levels can be crowdsourced in a few hours, and hence no one can actually justify collecting millions of end-user utterances every day to reach these metrics."
On Tuesday, the e-commerce giant began rolling out in the U.S. a new feature to all its Echo devices, Alexa Guard, that leverages the fact that its voice assistant is always listening to her surroundings.
When activated, Alexa Guard will use the device's integrated far-field microphones to continuously listen for glass breaks and alarms from a smoke or carbon monoxide detectors to protect your home or office, no wake word required.
Atherton Research Insights
Snips and more recently Google, which unveiled its next-generation Google Assistant at the Google I/O developer conference last week, are paving the way for a new kind of smart voice-assistant, designed with privacy in mind, that do not require anymore to upload user voice samples—or broken glass and alarms for that matter—to improve their performance and customer experience.
