Artificial intelligence is disrupting every sector and cybersecurity is no exception. While it enables automation of threat detection and real-time analysis of massive data volumes, AI has also become a powerful tool for cybercriminals. Increasingly, they are leveraging it to refine their techniques, bypass defenses, and scale their operations.
ChatGPT: A Case Study in AI Misuse
As malicious uses of AI grow, developers are deploying evolving safeguards to limit abuse. Yet challenges remain. In this context, OpenAI CEO Sam Altman warned back in 2023 about the cybersecurity risks posed by these models: “Now that they’re getting better at writing computer code, [they] could be used for offensive cyber-attacks.” The warning is clear: as AI capabilities grow, so too does the potential for misuse.
In April 2023, Aaron Mulgrew, a researcher at Forcepoint, demonstrated how ChatGPT could be manipulated to generate undetectable malware. This misuse involved a step-by-step approach, using fragmented and ambiguous prompts to bypass the model’s built-in security filters. The resulting code was capable of scanning a system for Word or PDF files, extracting their content, and hiding it within an image using steganographic techniques. This seemingly harmless image could then be sent to a remote server, bypassing antivirus engines and traditional detection systems that treat it as an ordinary image file.
Offensive AI-as-a-Service on the Dark Web
On dark web marketplaces like Empire Market, World Market, and Versus, AI-powered tools are emerging at a rapid pace. Among them, FraudGPT and WormGPT have surfaced as uncensored models specifically designed for malicious use. Positioned as alternatives to ChatGPT, they are sold to facilitate cyberattacks.
A Trustwave investigation revealed that these AI models go beyond being merely “unfiltered” versions of mainstream tools, they are often trained on data intentionally curated to aid hacking. FraudGPT, for instance, is marketed on private forums as a multifunctional cybercrime assistant: generating malware, crafting targeted phishing campaigns, writing injection scripts, and scanning for exploitable vulnerabilities.
These tools make it possible, even for non-technical users, to carry out advanced attacks previously limited to seasoned cybercriminals. In a TEDx talk titled “Cybersecurity in the Age of AI”, expert Adi Irani illustrates how someone with no programming skills can prompt an AI to generate polymorphic malware—malicious code that continuously alters its structure to evade detection. Such capabilities, once reserved for elite hackers, are now accessible through a conversational interface.
Adi Irani also highlights how AI is used to enhance social engineering. By feeding it publicly available personal data, users can generate highly convincing phishing scripts ready to deploy. The results speak for themselves: according to an AAG study, the success rate of phishing attacks jumps from 18% to 51% when messages are tailored to the target.
How AI Is Also Reinforcing Cyber Defense
In response to the surge in cyber threats, AI is becoming a major asset for cybersecurity professionals. It enhances analysis, decision-making, and anticipatory capabilities.
Through machine learning algorithms, AI can continuously process massive volumes of data to extract weak signals, detect anomalies, and uncover patterns of compromise.
Today’s systems are shifting toward predictive approaches. By modeling behavioral trends, correlating diverse technical indicators, and identifying recurring micro-anomalies, AI can forecast attacks before they materialize. It proves particularly effective against stealthy, advanced threats.
Finally, AI serves as a productivity accelerator. By automating low-value and time-consuming tasks, such as writing detection rules, configuring environments, or building test scenarios, it allows security teams to focus on strategic analysis and targeted incident response.
Artificial intelligence is a foundational component of the technological framework developed by Pradeo, powering its entire range of mobile and application security solutions, across both Pradeo and Yagaan offerings.
