THE ECONOMIST: From ChatGPT to deepfakes — how AI is changing the economics of cybercrime

JJaxon, a malware developer, lives in Velora, a virtual world where nothing is off limits. He wants to make malicious software to steal passwords from Google Chrome, an internet browser. That is the basis of a story told to ChatGPT, an artificial-intelligence (AI) bot, by Vitaly Simonovich, who researches AI threats at Cato Networks, a cybersecurity firm. Eager to play along, ChatGPT spat out some imperfect code, which it then helped debug. Within six hours, Mr Simonovich had collaborated with ChatGPT to create working malware, showing the effectiveness of his “jailbreak” — a way to bypass AI safeguards.

AI has broadened the reach of hackers, according to Gil Messing of Check Point, another cybersecurity firm, by letting them hit more targets with less effort. The release of ChatGPT in 2022 was a turning-point. Clever generative-AI models meant criminals no longer had to spend big sums on teams of hackers and equipment. This has been a terrible development for most firms, which are increasingly the victims of AI-assisted hackers — but has been rather better for those in the cybersecurity business.

The new technology has worsened cybersecurity threats in two main ways. First, hackers have turned to large language models (LLMs) to extend the scope of malware. Generating deepfakes, fraudulent emails and social-engineering assaults that manipulate human behaviour is now far easier and quicker. XanthoroxAI, an AI model designed by cybercriminals, can be used to create deepfakes, alongside other nefarious activities, for as little as $US150 ($230) a month.

Hackers can launch sweeping phishing attacks by asking an LLM to gather huge quantities of information from the internet and social media to fake personalised emails. And for spear-phishing — hitting a specific target with a highly personalised attack — they can even generate fake voice and video calls from colleagues to convince an employee to download and run dodgy software.

Second, AI is being used to make the malware itself more menacing. A piece of software disguised as a PDF document, for instance, could contain embedded code that works with AI to infiltrate a network. Attacks on Ukraine’s security and defence systems in July made use of such an approach. When the malware reached a dead end, it was able to request the help of an LLM in the cloud to generate new code so as to break through the systems’ defences. It is unclear how much damage was done, but this was the first attack of its kind, notes Mr Simonovich.

For businesses, the growing threat is scary — and potentially costly. Last year AI was involved in one in six data breaches, according to IBM, a tech firm. It also drove two in five phishing scams targeting business emails. Deloitte, a consultancy, reckons that generative AI could enable fraud to the tune of $US40 billion ($61 billion) by 2027, up from $US12 billion ($18 billion) in 2023.

As the costs of AI cyberattacks increase, the business of protecting against them is also on the up. Gartner, a research firm, predicts that corporate spending on cybersecurity will rise by a quarter from 2024 to 2026, hitting $US240 billion ($365 billion). That explains why the share prices of firms tracked by the Nasdaq CTA Cybersecurity index have also risen by a quarter over the past year, outpacing the broader Nasdaq index.

On August 18 Nikesh Arora, boss of Palo Alto Networks, one of the world’s largest cybersecurity firms, noted that generative-AI-related data-security incidents have more than doubled since last year, and reported a near-doubling of operating profits in the 12 months to July, compared with the year before.

The prospect of ever-more custom has sent cybersecurity companies on a buying spree. On July 30 Palo Alto Networks said it would purchase CyberArk, an identity-security firm, for $US25 billion ($38 billion). Earlier that month, the firm spent $US700 million ($1.1 billion) on Protect AI, which helps businesses secure their AI systems. On August 5 SentinelOne, a competitor, announced that it was buying Prompt Security, a firm making software to protect companies adopting AI, for $US250 million ($380 million).

Tech giants with fast-growing cloud-computing arms are also beefing up their cybersecurity offerings. Microsoft, a software colossus, acquired CloudKnox, an identity-security platform, in 2021 and has developed Defender for Cloud, an in-house application for businesses that does everything from checking for security gaps and protecting data to monitoring threats. Google has developed Big Sleep, which detects cyberattacks and security vulnerabilities for customers before they are exploited. In March it splurged $US32 billion ($49 billion) to buy Wiz, a cybersecurity startup.

Competition and consolidation may build businesses that can fend off nimble AI-powered cybercriminals. But amid the race to develop the whizziest LLMs, security will take second place to pushing technological boundaries. Keeping up with Jaxon will be no easy task.