Scammers are increasingly exploiting deepfake technology to deceive people online, and experts stress that awareness is essential to prevent these kinds of fraud.
An Indian textile magnate, SP Oswal, revealed that he was defrauded of 70 million rupees ($833,000) by cybercriminals who impersonated federal investigators and even the Chief Justice of India. The scammers, posing as officers from India’s Central Bureau of Investigation (CBI), contacted Oswal on August 28 and accused him of money laundering. Over the following two days, Oswal was kept under what appeared to be digital surveillance, with fraudsters ordering him to leave Skype open on his phone and threatening him with arrest. They even conducted a fake virtual court hearing with a deepfake version of the Chief Justice of India, DY Chandrachud, as the judge. Oswal, believing the scenario to be legitimate, transferred the funds after a fake verdict via Skype. He was later revealed to be the latest victim of a scam known as “digital arrest.”
What is a Digital Arrest?
A digital arrest is a new form of online fraud where scammers convince victims they are under “virtual arrest.” The victim is coerced into staying continuously connected to the scammer via video-conferencing software, effectively holding them hostage to their fraudulent demands. Similar to phishing, this type of cyberattack tricks individuals into revealing sensitive information, resulting in identity theft, financial loss, or data theft.
These scams have become more sophisticated with the use of AI-generated audio and video. The fraudster may manipulate video and audio to appear as a trustworthy figure, such as a judge or high-ranking official, and coerce the victim into following instructions under the threat of extreme consequences.
The use of video conferencing platforms makes these scams seem more legitimate, as scammers can use deepfake technology to appear as real people. AI-generated voice mimics further increase the illusion, making it difficult for the victim to detect the fraud.
Professor VS Subrahmanian of Northwestern University calls these scams a form of “spear-phishing,” where the attack is highly targeted, and victims are manipulated based on their specific circumstances. “Vishing” refers to voice phishing, and “phishing” can also occur via SMS.
The SP Oswal Incident and Other Digital Arrests
Oswal was targeted after receiving a call from a fraudster claiming there were financial irregularities linked to his account, which was supposedly involved in a case against Naresh Goyal, the former chairman of Jet Airways. The scammers used fake arrest warrants and Supreme Court documents to convince Oswal to transfer $833,000 into a specific account. After filing a police complaint, Oswal was able to recover $630,000 of the amount with the help of cybercrime officials. This case represents one of the largest recoveries for such scams in India.
Digital arrests have been increasingly prevalent since 2020, when many services shifted online due to the COVID-19 pandemic. In other cases, employees from the Raja Ramanna Advanced Technology Center and the National Buildings Construction Corporation were defrauded out of millions of rupees through similar scams.
The Rise of Deepfake Technology in Scams
Although deepfake technology has existed since 2015, its use in fraudulent schemes has grown more frequent and sophisticated due to advancements in machine learning and AI. Fraudsters now use deepfake software to embed anyone into a video or photo, even adding audio using deepfake AI tools. The result is a highly convincing impersonation that can take place via video-conferencing platforms such as Zoom, Skype, or Teams.
Some deepfake software requires just a brief sample of audio—10 seconds to a minute—to replicate a person’s voice with near-perfect accuracy, capturing nuances such as pitch, accent, and pauses. This makes the deepfake virtually indistinguishable from the real person’s speech. A prominent case involved fraudsters using AI-generated voice to steal €220,000 from a UK energy firm CEO.
The risks extend beyond financial fraud. Recently, a video-conference call was manipulated with a deepfake of Ukrainian Foreign Minister Dmytro Kuleba to influence political discussions, though no money was involved. This highlights how deepfake technology can be used to manipulate high-stakes political or diplomatic outcomes.
Why Are These Scams Rising?
The increasing use of deepfake AI tools for scams is linked to the rise of mobile-first internet users, particularly in countries like India. With many individuals relying on their phones as trusted sources of information, fraudsters are able to exploit their trust, making it difficult for them to initially question a call or video conference.
Professor Subrahmanian attributes India’s vulnerability to the lack of awareness about deepfakes, coupled with inadequate cybersecurity measures in the country’s telecommunication sector.
How Can Digital Arrests Be Prevented?
Deepfake technology relies on generative adversarial networks (GANs), which leave unique “artefacts” that can be detected by deepfake detection systems. These systems can identify the telltale signs of a deepfake, but as the technology advances, detection tools must evolve alongside it.
However, Professor Subrahmanian suggests that simply relying on detection software is insufficient. Building awareness about deepfakes is crucial, and a global initiative, similar to the EU’s General Data Protection Regulation (GDPR), may be necessary to address the issue.
International cooperation between law enforcement organizations, like Interpol, and better training for officials will be key in combating digital arrests and similar scams, he argues. Additionally, effective tools and measures must be in place to identify and mitigate these emerging threats to ensure that deepfake-based fraud does not continue to rise.
