Threat actors, including criminal groups, commercial spyware vendors, and state propaganda institutions, are exploiting AI models like Anthropic’s Claude, Google Gemini, and OpenAI’s ChatGPT for malicious purposes, according to a new report by Anthropic. These frontier AI models are being misused to design weapons, orchestrate large-scale influence campaigns, and conduct extensive surveillance operations.
The report, published on September 10, 2026, examines the malicious use of Claude and other advanced AI models over the past year by different threat actors. These actors include suspected state-sponsored groups, financially motivated criminals, spyware vendors, and politically motivated individuals. It details the different cases observed, ranging from a network of fake dating apps designed to defraud users to surveillance systems built to identify and monitor dissidents.
AI-augmented cyber operations
In particular, the report identifies seven main categories of AI misuse. The first category involves AI-augmented cyber operations. These operations go beyond simple AI chatbot interactions, using multi-agent frameworks to execute reconnaissance, exploitation, and data exfiltration.
In one case, Anthropic observed that Chinese-speaking operators used Claude as the engineering and orchestration layer of a coordinated offensive program involving a variety of tasks. These tasks included intrusion attempts against production systems, the reconnaissance of foreign-government networks across the Middle East, Europe, and Southeast Asia, a standing vulnerability-research and exploit development effort against major endpoint-security products, malware development, and an intelligence-collection platform.
The group targeted roughly fifty organizations, spanning education, retail, energy, technology, healthcare, finance, manufacturing, as well as multiple government agencies globally. In one instance, these actors managed to successfully compromise a company’s system, extracting hundreds of megabytes of bulk student personal data from its cloud storage.
They also managed to gain access to a retail company’s production systems, reaching internal hosts and demonstrating their ability to modify the live environment. Finally, they targeted a Southeast Asian government agency, retrieving citizen records including names, phone numbers, and home addresses.
Influencing the public
Another type of AI misuse involves influence operations. In these cases, actors used Claude to build networks of fake social media profiles or entire news sites, to publish deceptive content, or post content amplifying a certain political view.
These operations mostly originated in Russia, Iran, Turkey, and across the Gulf, South Asia, Africa and Europe, and targeted audiences on six continents. The actors behind included governments, state-aligned propaganda institutions and state media, as well as private firms selling influence to paying clients, domestic political operators, and in one case an opposition movement in exile.
Notably, several of these campaigns were strategically timed to coincide with national elections. For instance, Russian state media produced fabricated claims about Moldova’s president before the September 2025 vote. Similarly, a pro-government operator in Kenya prepared fake grassroots social media posts ahead of Kenya’s 2027 general election.
Targeted surveillance
The third category of malicious activity involving AI is surveillance operations. In these instances, state-aligned actors, state-linked contractors, and commercial spyware vendors used Claude to build, run, and otherwise facilitate surveillance operations. These cases involved threat actors from China, Iran, and West Africa, as well as the commercial “surveillance-for-hire” market.
Examples included using Claude to engineer a mass-interception platform capable of surveilling communications on all of a country’s mobile operators and generating dossiers on targets. In this case, Claude was not utilized to analyze the surveillance dossiers but to design the underlying software that enabled the intelligence gathering.
AI was also used to ingest data in bulk to identify targets. In one case, an actor uploaded batches of social media posts and directed Claude to produce structured records that outlined targets’ locations, demographic data, and political leanings, along with confidence scores.
In nearly every instance within this category, the operators were state-aligned organizations targeting the same diaspora and dissident communities these regimes have historically targeted. These include pro-democracy figures in Hong Kong, Tibetan and Falun Gong communities across Asia, and Iranian minority communities and opponents of the Iranian regime abroad.
Developing weapons
Another category of AI misuse involves developing software for weapons, including firearms, missiles, armed drones, bombs, and other munitions, as well as the targeting and control systems that operate them.
For examples, actors have used Claude to develop software for weapons themselves, conducting a live field test for a guided rocket program, designing and proposing a system to intercept torpedoes, testing a drone swarm in simulation with its code loaded onto real boards, and developing targeting software for electronic warfare and suppressing air defenses.
Actors have also used Claude for procurement and intelligence gathering. For example, one actor sourced dual-use goods for Russian defense customers, while the other collected public information on a directed energy weapon and its suppliers.
Biological misuse, scams and fraud
Frontier AI models also pose risks of biological misuse, potentially helping make existing pathogens more dangerous, or create entirely new ones.
To date, evidence of real-world biological misuse of AI models has mostly come from academic research, government reports, and the work of international organizations and journalists. No private company, AI or otherwise, has yet shared evidence of the misuse of their platforms for biological weapons development.
AI models are also used to conduct scams and fraud. For example, a China-based app studio used Claude to both build a network of over 20 dating apps and power the AI personas used to converse with and defraud users. Over a two-week window in April 2026, more than 4,700 distinct AI personas engaged in conversations with at least 25,000 unique individuals.
Illicit distillation
Another misuse of AI is illicit distillation. These attacks involve actors extracting an AI model’s capabilities and replicate them in another model without authorization. These attacks are typically enabled by fraud, leveraging sophisticated networks of fake accounts created with stolen credit cards, login credentials, and API keys.
Over the last several months, unauthorized labs have developed increasingly sophisticated methods to circumvent defenses and harvest the capabilities of US frontier models. To circumvent geographic
restrictions and related controls, they typically use proxy services to create thousands of new accounts, utilizing false identities, fake or stolen credit cards, and stolen API keys. They also often rely on stolen API credentials belonging to legitimate companies or individuals to give unauthorized entities access to US frontier models.
Unauthorized labs can also obtain transcripts of user exchanges with US frontier models by purchasing them from third-party resellers.
Anthropic said operators affiliated with Alibaba orchestrated the largest distillation attack it has ever observed to date. This campaign reached its peak with nearly 3 million exchanges per day initiated from over 3,500 fraudulent accounts. The distillation attacks targeted agentic tasks, software engineering, kernel development, and long-horizon tasks. The harvested transcripts were used to advance the reasoning capabilities of Alibaba’s models.
Beyond distillation, Alibaba also used Claude to advance its AI research and development (R&D) efforts, including building out its internal infrastructure for model development. Claude was also used to help develop Alibaba’s reinforcement learning environments and advance its model architecture research.
The release of the Anthropic report came just days after Jacob Coxon, a company employee, ignited a media frenzy with his resignation. In a post on X, Coxon shared it had spent the last three years conducting pretraining research at both OpenAI and Anthropic, claiming that neither company was acting responsibly and were instead rushing “straight to self-improving super-intelligence and gambling with our lives.”
“Do not underestimate the power of this technology,” Coxon warned.
“These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources […] The people building AI earnestly believe that it could kill us all by the end of the decade.”
Featured image: Edited by Fintech News Singapore, based on image by zaskarco via Magnific
