Do our phones listen to our conversations? The answer is complicated

The ability to say “Ok, Google” or “Hey, Siri” while sitting on our couch at home or driving to work, and to have our devices respond to our every command, is an undeniably useful perk in our increasingly digital lives. At last, tasks can be executed without having to place a single finger on the screen. But does this convenience come at the expense of our privacy? Do our cell phones really eavesdrop on us, even when we’re not knowingly giving them permission to listen?

The cybersecurity company that developed the NordVPN privacy software has proposed an experiment that, they say, allows users to test whether their phones are actively listening in the background to record what they hear and use it to cater advertising to individual users. To demonstrate, three NordVPN workers conducted the experiment themselves, placing their phones on a table near them, at a safe distance from each other, but within listening range of their respective owners. Then, each person spoke about a specific topic, using keywords like “Alaska” or “Volvo” and carrying on conversations with repeated references to those chosen terms.

The experiment participants repeated these sessions for three consecutive days, after which they analyzed the advertising content appearing on their devices. NordVPN has emphasized that the conclusions of the study “are not clear”; that is, they do not prove a direct correlation between the conversations and the content of the ads. But one of the test participants, who talked repeatedly about Volvo, did report that they were bombarded by advertising for the brand. This person did not even own a car, nor had they searched the Internet to buy a Volvo or any other kind of car. They did, however, occasionally follow Formula One racing events online.

Our mobile devices, essential components of our daily lives, have access to our private information and the capacity to track our digital activities and physical movements. To answer the question — do our cell phones listen to us? — we need to first understand how the listening capabilities of our devices actually work. Fernando Suárez, president of the Council of Colleges in Engineering Informatics, a Spanish nonprofit, explains that there are two general types of applications that might be listening in. “First, there are voice assistants like Siri and Alexa, which are programmed to always be attentive to our voice commands. Second, applications that we download may also have the ability to listen, depending on the permissions we grant them,” he says.

According to Suárez, if cell-phone users want to ensure that our data (and our conversations) are not being misused, we should make sure we know what permissions have been granted on our devices, both to the integrated system software as well as any third-party applications we install. These permissions can include giving apps unrestricted or partial access to our microphones and cameras. We can check and adjust these permissions in the privacy settings of our mobile devices. “It’s crucial to review and, if necessary, revoke permissions granted to applications that do not need such access,” Suárez advises.

“It’s crucial to review and, if necessary, revoke permissions granted to applications that do not need such access.”

Fernando Suárez, President of the Council of Colleges in Engineering Informatics

The next question is whether our devices are indeed spying on us. As NordVPN argues, the answer is essentially no, there is no conclusive evidence to suggest that phones are constantly recording or listening to our conversations. Our devices do collect a significant amount of data about us, however, even without needing to activate our microphones. The metadata from our Internet activity, which can range from our physical location, to the websites we visit, to who we interact with on social media, can reveal quite a lot about our lives.

But what about recording voice and key-term data? “Companies often claim that they don’t store this data, but will admit that they do use it to learn and improve their services. So, in a way, our devices are learning from us and adapting to better understand our requests,” concludes Suárez. The key difference is that this information remains completely anonymous; that is, it is not linked to the individual user’s account or profile. Google explains in its privacy policy that these recordings and their transcripts are stored in encrypted form, but that a small sampling may be heard at some point by human listeners.

Data and the possibility of spying

Google’s privacy disclaimer does not necessarily suggest that a human will be listening to everything spoken within earshot of a user’s cell phone, but that, on occasion (Google estimates roughly 0.2% of the total number of audio recordings analyzed), a team of people may listen to these recordings in order to improve the system’s learning capabilities. This information is anonymous, and is not linked to a specific person.

Responding to a request for comment from EL PAÍS, Google categorically denied that these audio recordings might be used for any other purpose: “Google does not use the background sounds recorded by any device for advertising purposes,” the technology giant confirmed. “The belief that devices listen to us [in order to individually cater advertising] has been widely disproven.”

The cybersecurity company Wandera conducted an experiment similar to the one performed by NordVPN, isolating two cell phones (an Android and an iPhone) in a room for 30 minutes and playing recordings of cat- and dog food advertising. The idea was to analyze whether any correlation could be identified between the subsequent advertising that appeared on the devices and the content of the audio played during the test. The answer was a resounding no.

The reality is that it is impossible to have a secure system in which the user does not give up some privacy in exchange for enjoying the service; there will always be a toll to pay. The platforms compensate for this complex balance by guaranteeing security measures to protect the information provided by the user. To this end, Apple, for example, has incorporated an extra layer of security in its iPhone software: as Julio César Fernández, academic director at Apple Coding Academy, explains, “when a developer wants to use the microphone in iOS, permission is requested from the user, and if an out-of-the-ordinary use of the microphone is detected, Apple rejects the application.”

But perhaps the method that offers the best guarantee is the feature that warns users when their device is listening: “When an application is using the microphone, an indicator light will appear at the top of the screen,” says Fernández “Therefore, on an iPhone without a jailbreak [a patch that removes some of the limitations that Apple imposes on the software of its mobile devices], it is impossible to activate the microphone without the user’s knowledge and consent.” This applies to applications, but how does the iPhone handle the active listening of its virtual assistant, Siri? “If the user gives permission, Apple applies differential privacy algorithms to protect the user’s identity, so that the source of the recording cannot be identified,” Fernández explains. He recommends, however, that users always keep their device’s software up to date to avoid possible security flaws that might be exploited by hackers, who could then gain unauthorized access to the microphone.

So, knowing all this, do cell phones listen to us or not? The answer is too complicated for easy monosyllabic assurances. Mobile devices certainly have the ability to listen, offering users the added value of a virtual assistant, which can be activated by voice alone; but this does not mean that someone is always, or ever, listening to our commands and conversations without our consent. Operating systems and applications are designed with built-in controls that offer certain security and privacy guarantees. But, at the end of the day, the user — even assuming those privacy barriers work — should know that there is always a remote possibility that one of their conversations or commands could be heard by someone working to improve the quality of the systems they use.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition