Content
summary Summary

Cybercriminals are increasingly using generative AI tools such as ChatGPT or WormGPT, a dedicated malware model, to send highly convincing fake emails to organizations, allowing them to bypass security measures.

Ad

A new wave of highly convincing fake emails is hitting unsuspecting employees. That's according to British computer hacker Daniely Kelley, who has been researching WormGPT, an AI tool optimized for cybercrimes such as Business Email Compromise (BEC). Kelley is referring specifically to his observations on underground forums.

In addition, special prompts known as "jailbreaks" are exchanged to manipulate models such as ChatGPT to generate output that could include the disclosure of sensitive information or the execution of malicious code.

"This method introduces a stark implication: attackers, even those lacking fluency in a particular language, are now more capable than ever of fabricating persuasive emails for phishing or BEC attacks," Kelley writes.

Ad
Ad

AI-generated emails are grammatically correct, which Kelley says makes them more likely to go undetected, and easy-to-use AI models have lowered the barrier to entry for attacks.

WormGPT is an AI model optimized for online fraud

WormGPT is an AI model specifically designed for criminal and malicious activity that is shared on popular online cybercrime forums. It is marketed as a "blackhat" alternative to official GPT models and advertised with privacy protection and "fast money".

Similar to ChatGPT, WormGPT can create convincing and strategically sophisticated emails. This makes it a powerful tool for sophisticated phishing and BEC attacks. Kelley describes it as ChatGPT with "no ethical boundaries or restrictions".

WormGPT is based on the open-source GPT-J model, which approaches the performance of GPT-3 and can perform textual tasks similar to ChatGPT, as well as write or format simple code. The WormGPT derivative is said to have been trained with additional malware datasets, although the author of the hacking tool does not disclose these.

Ein Screenshot mit weißem Text auf rotem Hintergrund.
A phishing email generated with WormGPT. | Image: Daniel Kelley, via SlashNext

Kelley tested WormGPT on a phishing email designed to trick a customer service representative into paying an urgent bogus bill. The sender of the email was the CEO of the targeted company (see screenshot).

Recommendation

Kelley calls the results of the experiment "unsettling" and the fraudulent email generated "remarkably persuasive, but also strategically cunning." Even inexperienced cybercriminals could pose a significant threat with a tool like WormGPT, Kelley writes.

The best protection against AI-based BEC attacks is prevention

As AI tools continue to proliferate, new attack vectors will emerge, making strong prevention measures essential, Kelley says. Organizations should develop BEC-specific training and implement enhanced email verification measures to protect against AI-based BEC attacks.

For example, Kelley cites alerts for emails from outside the organization that impersonate managers or suppliers. Other systems could flag messages with keywords such as "urgent," "sensitive," or "wire transfer," which are associated with BEC attacks.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Cybercriminals are using generative AI applications such as OpenAI's ChatGPT and WormGPT to create convincing fake emails for corporate fraud attacks.
  • WormGPT is an AI model designed specifically for criminal and malicious activity and is promoted as a "blackhat" alternative to official GPT models. It has allegedly been fine-tuned using malware data.
  • To protect against AI-based fraud attacks, organizations should develop BEC-specific training and implement enhanced email verification measures.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.