The software that can detect if your photos have been faked

A talent for deceit and a few photos were enough for Israeli scammer Simon Leviev to make millions of dollars on Tinder. He seemed to live a life of luxury in a mansion overlooking the sea. Photos showed him flying on private jets, driving luxury cars and vacationing in fabulous hotels. The photos he shared on the dating app looked real. However, combined with a dose of romance, they were enough to ruin dozens of women who didn’t think twice about lending him money that he never paid back. This is the story told in the Netflix documentary, The Tinder Swindler, and also in lectures by Fernando Pérez-González, a professor at Spain’s University of Vigo, who leads the research team that created a software application called Fawrensian. The application detects whether digital photo files have been manipulated and can analyze more than one million documents a day. The technology, which is aimed at combatting cyberfraud, has been licensed by a number of companies (including some in Silicon Valley) to conduct forensic analyses of multimedia content such as personal documents.

It all started 15 years ago with a question. “We had been working on image watermarking, which basically consists of hiding information in photographs to protect copyrights and the like,” says Pérez-González in his university office. The team then started thinking about how they could determine if a photograph had been taken by a particular camera. “We’re talking about a specific camera, not just the camera model.” They discovered that a camera’s sensors have imperfections that are reflected in the images, but are undetectable to the naked eye. “It’s like a fingerprint.” That fingerprint can be extracted from a group of photos, including those taken with a cellphone camera. “Then, you can identify the camera that took the photo.”

Fawrensian can be used for forensic applications. For example, if the police seize a pedophile’s hard drive, they can determine how many different cameras have taken all the images stored on the hard drive. It’s important to obtain several photos because some low quality images will not allow the extraction of a fingerprint. This process translates into different brackets of probability, similar to those produced by DNA tests to determine whether a genetic sample belongs to an individual. “It’s pretty amazing. Some people call it photographic ballistics, because it’s very similar to ballistic tests that can identify the gun that fired a specific bullet.”

Forensic tools have adapted to market demand over time. “We started out thinking about how to identify fake news, but the real money is in Know Your Customer (KYC) processes, such as when a lender asks for an image of your ID card when you apply for a loan. There is a lot at stake here and companies need new tools.” Fawrensian looks for changes to the properties of digital documents in order to detect inconsistencies. There are many examples of altered images circulating on Twitter, such as a discontinued Benetton advertising campaign that used a photo of Pope Benedict XVI kissing a Muslim imam on the mouth. “When compressing a jpg image file, for example, a number of very specific properties are recorded in the image. We can detect traces of that double image compression. Many social media platforms re-compress the photos we all upload, and there’s nothing wrong with that. But if you send a photo of your ID card to a bank, there should be no double compression. We are able to detect and warn you when this happens.”

A filter for Twitter?

David Vázquez, a researcher with the Fawrensian team, demonstrates the software to EL PAÍS. He changes one number on a photo of his ID card, which takes about five seconds. The change is imperceptible. Vázquez then uses Fawrensian to analyze the image and convert it into a heat map showing the probability of manipulation for each pixel in the image. The heat map clearly reveals the change to the ID card. Vázquez grabs some other images from Twitter posts and processes them through Fawrensian. The program soon detects the manipulation of a photo posted about the 2019 pro-independence protests in Catalonia. One would think that many people would pay to use Fawrensian to check Tinder or Twitter photos, but there is no plan to market it to individual users. “It would be much more difficult for us to manage. How often would each customer use it? They probably wouldn’t even be willing to pay for it. And we would have to staff a customer service department. It would be a different business model than the one we have now,” say the Fawrensian team.

Developed with financial support from the Galician regional government in Spain, Fawrensian is currently licensed only to businesses. Five people from the University of Vigo’s Center for Telecommunication Technology Research (Atlanttic) developed and support the application. The team hopes to use product revenue to repay their financial backers. “Our objective is for people to use this product so that we can continue to solve other problems. We hope that the university will continue to market the technologies it develops.” They believe that in a few years, the university will be able to recoup the $377,000 it invested in developing Fawrensian.