Malicious AI Models Surge on Dark Web: WormGPT, FraudGPT, and the Rise of Unrestricted Cybercrime Tools

The emergence of malicious AI models on the dark web marks a significant shift in the landscape of cybercrime. These custom or fine-tuned language models, stripped of safety filters, are being sold on underground forums to facilitate phishing emails, generate malware, and automate fraud. Notable names like WormGPT and FraudGPT have come to symbolize this trend, representing a category of tools that security teams must understand to effectively combat these threats.

What Is Dark Web AI? Defining the Threat Category

“Dark web AI” encompasses large language models that have been intentionally designed or modified to eliminate the ethical guardrails present in mainstream AI tools such as ChatGPT or Gemini. These models are marketed to cybercriminals via underground forums and Telegram channels. They are not entirely new architectures; rather, they are often fine-tuned versions of open-source models, rebranded with a criminal-facing interface and a subscription price tag.

Uncensored LLMs vs. Jailbroken Models

There are two primary approaches within this category. Some tools, like the original WormGPT, are trained from the ground up on malware and phishing data, meaning that rejecting harmful requests was never part of their design. Others operate as wrapper services, routing prompts through a jailbroken session of a mainstream model, employing prompt tricks to bypass safety layers. Both methods yield an assistant devoid of content moderation, capable of generating unrestricted outputs.

Why Threat Actors Moved to AI

The appeal of these tools lies in their speed and polish, rather than mere novelty. Previously, crafting convincing phishing emails or malicious scripts required either language or coding skills. With a capable text-generation model, even a non-native English speaker can produce a grammatically correct business email compromise (BEC) lure in seconds. Security researchers at Huntress have noted that this advancement effectively eliminates classic phishing red flags, such as poor grammar and awkward phrasing, which employees have been trained to recognize.

A Field Guide to Known Malicious AI Models

Since mid-2023, a small ecosystem of branded “dark LLMs” has emerged, each marketed with unique claims but built on the same premise: an AI assistant without ethical limits, sold as a criminal product.

WormGPT

WormGPT is recognized as the first commercialized malicious LLM, appearing on hacker forums in 2023 as a “blackhat alternative” to ChatGPT. It was developed by fine-tuning the open-source GPT-J model on malware and phishing data, marketed on a subscription basis ranging from $60 to $700. The original developer shut down the project in August 2023 due to media and security research scrutiny. However, the name “WormGPT” has since been appropriated by various copycat services, transforming it into a brand rather than a singular tool.

FraudGPT

FraudGPT explicitly markets itself as a tool for “fraudsters, hackers, scammers, and like-minded individuals.” It is sold by a vendor claiming verified status across several dark web marketplaces. Its feature set extends beyond phishing emails to include building phishing pages, writing malicious code, locating leaked data, and identifying non-VBV payment card numbers for fraud. Reports from SecureOps indicate that the seller has claimed over 3,000 confirmed sales across subscription tiers priced at approximately $1,700 per year, demonstrating its operation as a commercial product.

Other Variants

Following the emergence of WormGPT and FraudGPT, a rotating array of similarly branded tools, including EvilGPT, WolfGPT, and VenomGPT, have appeared on dark web forums. These tools generally offer the same core functionality: an “unrestricted” chatbot for generating phishing content, scam scripts, or code assistance. Security researchers note that many of these are rebrands or resellers, rather than distinct models, and a significant portion of the underground AI product market consists of tools that underdeliver on their marketing or are outright scams targeting criminals.

Documented Decline and Churn

Researchers have observed that these subscription-based tools often have a short lifespan. Within six months of the initial hype surrounding WormGPT and FraudGPT, threat intelligence teams noted a shift among criminals toward repurposing openly available, uncensored models instead of paying for branded products. Open-source alternatives can match or exceed the output of branded tools at no cost, indicating that the “dark web AI” market is both real and volatile.

How These Tools Are Used in Real Attacks

The primary application of malicious AI tools is the generation of convincing, error-free phishing and business email compromise content at a scale and speed that manual writing cannot achieve. A single operator can produce hundreds of unique, polished phishing emails in the time it once took to draft one by hand, undermining detection methods that rely on recognizing repeated templates.

Phishing and BEC Email Generation

This use case remains the most well-documented. Researchers from SlashNext tested WormGPT by having it draft an email intended to persuade an accounts payable employee to pay a fraudulent invoice. The results were described as “unsettling” in their persuasiveness, highlighting the core commercial appeal of tools in this category.

Malware and Code Assistance

Several of these tools advertise capabilities for generating or obfuscating code. Demo material for FraudGPT has reportedly shown it producing functional code for fake bank login pages. While the output quality can vary and is not always production-ready, it significantly reduces the effort and skill required to initiate such attacks.

Social Engineering Scripts and Impersonation

Beyond email, these models can generate conversational scripts that impersonate employees, vendors, or support staff in real-time, enabling scam calls, fake support chats, and SMS phishing (smishing) campaigns.

Fraud Automation

Some tools extend into locating leaked data, identifying usable stolen payment card numbers, and constructing supporting infrastructure like phishing pages. This turns the model into a component of a broader fraud pipeline rather than a standalone chatbot.

Why They’re Dangerous: Removing the Guardrails

The primary danger posed by these tools lies not in new capabilities but in the deliberate removal of safety layers that legitimate AI providers incorporate by default. Mainstream models are designed to reject requests for phishing content, malware, or fraud scripts; malicious AI tools are engineered to ensure such requests are never denied.

No Ethical Filters

While legitimate providers invest in content moderation, abuse monitoring, and regulatory compliance, these tools are marketed with the explicit promise of lacking such safeguards. FraudGPT’s advertising describes it as a “bot without limitations, rules, and boundaries.”

Subscription and Dark-Market Business Models

These tools operate as genuine businesses, complete with pricing tiers, customer support claims, and marketplace listings across dark web markets. This commercial structure indicates a real demand that sustains recurring revenue, ensuring that the tools will continue to evolve rather than disappear.

Rapid Iteration and Rebranding Cycles

When one tool attracts too much law enforcement or media attention and is shut down, the underlying concept does not vanish; the brand name is often reused by unrelated operators, or the same functionality reappears under a new name. Defenders focused on blocking specific tools will always be a step behind the next rebrand.

2024–2026 Trends: The Evolving Landscape

The dark web AI landscape is shifting from branded subscription products toward more decentralized, harder-to-track alternatives. As open-source language models become more capable and easier to fine-tune, criminals increasingly prefer to repurpose freely available models rather than pay for packaged tools like WormGPT or FraudGPT, as the results are comparable at no cost.

New Entrants and Rebrands

The era of WormGPT and FraudGPT has given way to a longer list of similarly named tools. Researchers have documented instances of threat actors repurposing legitimate security-research tools originally designed to assist red teams in testing defenses for offensive purposes.

Law Enforcement and Industry Scrutiny

Agencies such as Europol and the FBI have publicly acknowledged the rise of AI-enabled cybercrime, contributing to shutdowns like WormGPT’s in 2023. Ironically, even these criminal tools are not immune to security failures; a 2026 data leak reportedly exposed nearly 19,000 WormGPT user accounts, including emails and payment metadata, transforming the anonymity sought by buyers into a new identification risk.

Healthy Skepticism in the Research Community

Not every claim regarding AI-powered cybercrime holds up. Some widely cited statistics, such as the assertion that 80% of ransomware is AI-enabled, have been debunked. Researchers monitoring underground forums have found that many criminals themselves express skepticism about these tools, describing the branded AI product market as largely low-quality or outright scams.

How Organizations Can Detect and Defend Against AI-Generated Attacks

Defending against AI-generated attacks requires a shift away from detection methods that rely on identifying human error, as these tools are designed to eliminate such indicators. The first step is acknowledging that grammar and tone are no longer reliable phishing indicators.

Recognize AI-Written Phishing Red Flags

Traditional advice to watch for typos, awkward phrasing, and generic greetings is becoming less effective against AI-generated lures. Updated training should focus on behavioral cues, such as unexpected urgency, unusual payment or credential requests, and discrepancies between a message’s claimed sender and normal communication patterns, verified through out-of-band checks.

Monitor for Brand and Credential Exposure on the Dark Web

These tools are often used to weaponize leaked data, combining stolen credentials and breached personal information into personalized lures. Continuous dark web monitoring for exposed employee credentials, brand mentions, and leaked customer data can significantly reduce the raw material these attacks depend on. Identifying exposure before it is utilized in a campaign is often more effective than attempting to catch the resulting email after it has been sent.

Adapt Employee Training

Awareness programs need to reflect the current landscape of AI-generated phishing, which is now polished, personalized, and devoid of traditional warning signs. Coupling this training with strong authentication, email filtering tuned to behavioral anomalies, and network segmentation can limit damage even when a well-crafted lure manages to penetrate defenses.

For further insights into the evolving landscape of cybersecurity threats, visit www.dexpose.io.

Keep reading for the latest cybersecurity developments, threat intelligence and breaking updates from across the Middle East.