Alexa will employ AI to get promoted to butler of the household

Voice assistants like Siri, Alexa, Google Assistant and Bixby are good at playing music, answering simple questions, displaying digital photos, setting timers and alarms, and turning smart switches on and off. But smartphones can also do all these things, so voice assistants need to evolve if they want to survive. Voice commerce (making purchases using voice commands) could be the next step for voice assistants — Nielsen predicts it will account for 2% of all retail buying by 2025. In the meantime, tech giants like Google, Microsoft, Samsung, Meta and several Elon Musk companies are all focusing on how generative artificial intelligence (AI) can enhance their products. But Amazon’s strategy is to use AI to transform Alexa into a home hub for household appliances, a computer for everyday use and a butler who can perform routine tasks for its masters.

Alexa has been integrated into over 300 million devices sold by Amazon and others. Despite this massive penetration, some technology writers say Alexa is the company’s greatest failure. Vishal Sharma, vice president of Alexa’s artificial intelligence organization, refutes this claim. “It has never been a failure. If you look at the usage statistics or the number of connected devices, you will see it has grown.” According to Amazon, Alexa interactions grew globally by over 30% last year, and 50% of users made purchases with Alexa.

Alexa has never been a failure; it has grown in usage and in number of connected devices

Vishal Sharma, vice president at Amazon and head of artificial intelligence for Alexa

EL PAÍS attended an Amazon press briefing in which Sharma said the company will double down on using artificial intelligence to turn Alexa into a nerve center for the home. Dave Limp, Amazon’s VP for devices and services, said, “Cell phones have driven most of the innovation in the industry — miniaturization of cameras, new sensors, batteries… But the home, where we spend much of our time, has been ignored.”

“Homes are incredibly manual. The industry has been automating other areas, but not the home. So we’ve developed this vision of a home that will serve its inhabitants by doing tasks and being proactive [through learning routines]. This is what we call ambient intelligence,” said Limp.

Vishal Sharma says automation using verbal commands hasn’t matured because of the complex task of distinguishing between voices and filtering out extraneous noise so a reliable response can be produced. If a voice assistant is asked to turn on a light, it needs to turn on the right one. The other key is to learn routines. The assistant needs to recognize that a sequence of repeated actions is a habitual pattern, and then group them under a single command.

Cell phones have driven most of the innovation in the industry. But the home, where we spend much of our time, has been ignored

Dave Limp, VP for devices and services at Amazon

Marja Koopmans, who heads up Amazon’s smart home lab in Seattle, Washington, demonstrates how it works. When she said, “Alexa, I’m leaving,” the assistant locked the door and turned off all lights and devices except for the ones programmed to run when people are away, like the vacuum cleaner. When it was time to wake up in the morning, Alexa turned on the lights, provided the weather forecast, read the day’s schedule, and turned on the shower and coffee maker.

Koopmans says there are four main reasons for having a smart home: it facilitates tasks; provides entertainment (music, TV, games, etc.); offers security for people (mainly for older adults and children) and assets (remote monitoring of homes); and conserves energy by shutting down devices that are not in use.

Vishal Sharma believes integrating all these devices with a single assistant offers a distinct advantage over smartphones that can perform similar functions. “You don’t have to pick up your phone and click 1,800 things or scroll all over the place. It just works without you having to worry about the technology behind it.”

Our strategy is not so much to make money on the devices, but to make money on their use

Vishal Sharma

The cost of this vision will vary depending on the home hub device ($100-$275), the smart plugs used (starting at $10), and the compatible appliances and devices integrated with Alexa, like the Fire TV Stick or Cube. “Our strategy is not so much to make money on the devices, but to make money on their use,” said Sharma.

One of the biggest obstacles to a connected life is data security and privacy. Mattia Epifani, a digital forensic analyst and instructor at the SANS Institute, told MIT Technology Review evidence of a crime can come from many places. “It can be a location. It can be a message. It can be a picture. It can be anything. Maybe it can also be the heart rate of a user or how many steps the user took. And all these things are basically stored on electronic devices.” Leila Rouhi, VP of trust and privacy at Amazon, says that it’s a priority issue and acknowledges the company has made mistakes. “If we make a mistake, we break user trust, and it’s incredibly difficult for us to get it back. That’s why it’s really important for us to think about privacy and trust, which means investing in inclusive artificial intelligence to ensure this.” Rouhi says Amazon’s key principles are transparency, never selling users’ personal information, and always giving them a choice to opt out of recording data, images and sounds.

We never sell users’ personal information

Leila Rouhi, VP of trust and privacy at Amazon

This commitment to privacy conflicts with the constant need for devices to learn through experience. To distinguish between the voices of a child and an older adult, between a woman and a man, or between different accents of the same language, machines have to learn from sample recordings. Amazon says its machine learning processes are always supervised and regulated by continuous controls.

Sharma insists that security and privacy are taken “very seriously… We are transparent with what happens under the covers, and we always offer the ability to permanently delete anything.” Some Amazon devices will only store information internally or won’t record any at all unless a user intentionally enables it. But using data is critical, says Sharma, who offers an analogy to explain. “When you talk to a friend, you usually want to remember what you said, especially if the friend is helping you with something.” But Sharma insisted: “We will move heaven and earth to protect privacy and ensure security.”

The AI race

Every tech giant is working to deploy AI across the board, but the race has sped up recently as the sector begins to show signs of recovery. Microsoft invested $10 billion in OpenAI, the company behind the ChatGPT chatbot that it integrated into its Bing search engine. But this is only the first step for Microsoft. CEO Satya Nadella says they will incorporate artificial intelligence into other products to open up “a new era of computing”.

In March, South Korean consumer electronics giant Samsung rocked the tech world by saying it was considering replacing Google with Bing as the default search engine on its devices. Google’s response was to accelerate development of new artificial intelligence systems and upgrade existing ones with AI features. Like Amazon, Google’s Magi Project aims to offer users a personalized experience and anticipate their needs.

Google hastily released Bard shortly after ChatGPT was launched, but artificial intelligence is not new to the company. Its DeepMind subsidiary has been developing AI applications for years in language models, autonomous cars, search engines, music playback and computer programming.

Even Elon Musk (Tesla, Twitter and SpaceX) has joined the AI race while simultaneously warning about the potential dangers of the technology. According to The New York Times, Musk intends to start a new artificial intelligence company called X.AI to compete with OpenAI’s ChatGPT.

Through its AWS cloud computing platform, Amazon has made its generative large language models (grouped under the name Titan) available to customers, as well as a cloud computing service called Bedrock to create and scale generative AI applications for on-demand creation of text, images, audio and data.

Meta unveiled a new open source tool in May called ImageBind that the company says can match objects in a photograph to their sound, 3D shape or motion, and create images from noises. In the future, Meta hopes to endow the machine with other senses, like touch and smell, that will approximate human capabilities. Another open source AI tool from Meta is LLM Meta AI, which aims to compete with ChatGPT and similar applications.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition