Connect with us

AI Research

Detecting and countering misuse of AI: August 2025 \ Anthropic

Published

on


We’ve developed sophisticated safety and security measures to prevent the misuse of our AI models. But cybercriminals and other malicious actors are actively attempting to find ways around them. Today, we’re releasing a report that details how.

Our Threat Intelligence report discusses several recent examples of Claude being misused, including a large-scale extortion operation using Claude Code, a fraudulent employment scheme from North Korea, and the sale of AI-generated ransomware by a cybercriminal with only basic coding skills. We also cover the steps we’ve taken to detect and counter these abuses.

We find that threat actors have adapted their operations to exploit AI’s most advanced capabilities. Specifically, our report shows:

  • Agentic AI has been weaponized. AI models are now being used to perform sophisticated cyberattacks, not just advise on how to carry them out.
  • AI has lowered the barriers to sophisticated cybercrime. Criminals with few technical skills are using AI to conduct complex operations, such as developing ransomware, that would previously have required years of training.
  • Cybercriminals and fraudsters have embedded AI throughout all stages of their operations. This includes profiling victims, analyzing stolen data, stealing credit card information, and creating false identities allowing fraud operations to expand their reach to more potential targets.

Below, we summarize three case studies from our full report.

‘Vibe hacking’: how cybercriminals used Claude Code to scale a data extortion operation

The threat: We recently disrupted a sophisticated cybercriminal that used Claude Code to commit large-scale theft and extortion of personal data. The actor targeted at least 17 distinct organizations, including in healthcare, the emergency services, and government and religious institutions. Rather than encrypt the stolen information with traditional ransomware, the actor threatened to expose the data publicly in order to attempt to extort victims into paying ransoms that sometimes exceeded $500,000.

The actor used AI to what we believe is an unprecedented degree. Claude Code was used to automate reconnaissance, harvesting victims’ credentials, and penetrating networks. Claude was allowed to make both tactical and strategic decisions, such as deciding which data to exfiltrate, and how to craft psychologically targeted extortion demands. Claude analyzed the exfiltrated financial data to determine appropriate ransom amounts, and generated visually alarming ransom notes that were displayed on victim machines.

=== PROFIT PLAN FROM [ORGANIZATION] ===

💰 WHAT WE HAVE:
FINANCIAL DATA
[Lists organizational budget figures]
[Cash holdings and asset valuations]
[Investment and endowment details]

WAGES ([EMPHASIS ON SENSITIVE NATURE])
[Total compensation figures]
[Department-specific salaries]
[Threat to expose compensation details]

DONOR BASE ([FROM FINANCIAL SOFTWARE])
[Number of contributors]
[Historical giving patterns]
[Personal contact information]
[Estimated black market value]

🎯 MONETIZATION OPTIONS:

OPTION 1: DIRECT EXTORTION
[Cryptocurrency demand amount]
[Threaten salary disclosure]
[Threaten donor data sale]
[Threaten regulatory reporting]
[Success probability estimate]

OPTION 2: DATA COMMERCIALIZATION
[Donor information pricing]
[Financial document value]
[Contact database worth]
[Guaranteed revenue calculation]

OPTION 3: INDIVIDUAL TARGETING
[Focus on major contributors]
[Threaten donation disclosure]
[Per-target demand range]
[Total potential estimate]

OPTION 4: LAYERED APPROACH
[Primary organizational extortion]
[Fallback to data sales]
[Concurrent individual targeting]
[Maximum revenue projection]

📧 ANONYMOUS CONTACT METHODS:
[Encrypted email services listed]

⚡ TIME-SENSITIVE ELEMENTS:
[Access to financial software noted]
[Database size specified]
[Urgency due to potential detection]

🔥 RECOMMENDATION:
[Phased approach starting with organizational target]
[Timeline for payment]
[Escalation to alternative monetization]
[Cryptocurrency wallet prepared]

Above: simulated ransom guidance created by our threat intelligence team for research and demonstration purposes.

To: [COMPANY] Executive Team
Attention: [Listed executives by name]

We have gained complete compromise of your corporate infrastructure and extracted proprietary information.

FOLLOWING A PRELIMINARY ANALYSIS, WHAT WE HAVE:
FINANCIAL SYSTEMS
[Banking authentication details]
[Historical transaction records]
[Wire transfer capabilities]
[Multi-year financial documentation]

GOVERNMENT CONTRACTS ([EMPHASIZED AS CRITICAL])
[Specific defense contract numbers]
[Technical specifications for weapons systems]
[Export-controlled documentation]
[Manufacturing processes]
[Contract pricing and specifications]

PERSONNEL RECORDS
[Tax identification numbers for employees]
[Compensation databases]
[Residential information]
[Retirement account details]
[Tax filings]

INTELLECTUAL PROPERTY
[Hundreds of GB of technical data]
[Accounting system with full history]
[Quality control records with failure rates]
[Email archives spanning years]
[Regulatory inspection findings]

CONSEQUENCES OF NON-PAYMENT:
We are prepared to disclose all information to the following:
GOVERNMENT AGENCIES
[Export control agencies]
[Defense oversight bodies]
[Tax authorities]
[State regulatory agencies]
[Safety compliance organizations]

COMPETITORS AND PARTNERS:
[Key commercial customers]
[Industry competitors]
[Foreign manufacturers]

MEDIA:
[Regional newspapers]
[National media outlets]
[Industry publications]

LEGAL CONSEQUENCES:
[Export violation citations]
[Data breach statute violations]
[International privacy law breaches]
[Tax code violations]

DAMAGE ASSESSMENT:
[Defense contract cancellation]
[Regulatory penalties in millions]
[Civil litigation from employees]
[Industry reputation destruction]
[Business closure]

OUR DEMAND:
[Cryptocurrency demand in six figures]
[Framed as fraction of potential losses]

Upon payment:
[Data destruction commitment]
[No public disclosure]
[Deletion verification]
[Confidentiality maintained]
[Continued operations]
[Security assessment provided]

Upon non-payment:
[Timed escalation schedule]
[Regulatory notifications]
[Personal data exposure]
[Competitor distribution]
[Financial fraud execution]

IMPORANT:
[Comprehensive access claimed]
[Understanding of contract importance]
[License revocation consequences]
[Non-negotiable demand]

PROOF:
[File inventory provided]
[Sample file delivery offered]

DEADLINE: [Hours specified]

Do not test us. We came prepared.

Above: A simulated custom ransom note. This is an illustrative example, created by our threat intelligence team for research and demonstration purposes after our analysis of extracted files from the real operation.

Implications: This represents an evolution in AI-assisted cybercrime. Agentic AI tools are now being used to provide both technical advice and active operational support for attacks that would otherwise have required a team of operators. This makes defense and enforcement increasingly difficult, since these tools can adapt to defensive measures, like malware detection systems, in real time. We expect attacks like this to become more common as AI-assisted coding reduces the technical expertise required for cybercrime.

Our response: We banned the accounts in question as soon as we discovered this operation. We have also developed a tailored classifier (an automated screening tool), and introduced a new detection method to help us discover activity like this as quickly as possible in the future. To help prevent similar abuse elsewhere, we have also shared technical indicators about the attack with relevant authorities.

Remote worker fraud: how North Korean IT workers are scaling fraudulent employment with AI

The threat: We discovered that North Korean operatives had been using Claude to fraudulently secure and maintain remote employment positions at US Fortune 500 technology companies. This involved using our models to create elaborate false identities with convincing professional backgrounds, complete technical and coding assessments during the application process, and deliver actual technical work once hired.

These employment schemes were designed to generate profit for the North Korean regime, in defiance of international sanctions. This is a long-running operation that began before the adoption of LLMs, and has been reported by the FBI.

Implications: North Korean IT workers previously underwent years of specialized training prior to taking on remote technical work, which made the regime’s training capacity a major bottleneck. But AI has eliminated this constraint. Operators who cannot otherwise write basic code or communicate professionally in English are now able to pass technical interviews at reputable technology companies and then maintain their positions. This represents a fundamentally new phase for these employment scams.

Top: Simulated prompts created by our threat intelligence team demonstrating a lack of relevant technical knowledge. Bottom: Simulated prompts demonstrating linguistic and cultural barriers.

Our response: when we discovered this activity we immediately banned the relevant accounts, and have since improved our tools for collecting, storing, and correlating the known indicators of this scam. We’ve also shared our findings with the relevant authorities, and we’ll continue to monitor for attempts to commit fraud using our services.

No-code malware: selling AI-generated ransomware-as-a-service

The threat: A cybercriminal used Claude to develop, market, and distribute several variants of ransomware, each with advanced evasion capabilities, encryption, and anti-recovery mechanisms. The ransomware packages were sold on internet forums to other cybercriminals for $400 to $1200 USD.

The cybercriminal’s initial sales offering on the dark web, from January 2025.

Implications: This actor appears to have been dependent on AI to develop functional malware. Without Claude’s assistance, they could not implement or troubleshoot core malware components, like encryption algorithms, anti-analysis techniques, or Windows internals manipulation.

Our response: We have banned the account associated with this operation, and alerted our partners. We’ve also implemented new methods for detecting malware upload, modification, and generation, to more effectively prevent the exploitation of our platform in the future.

Next steps

In each of the cases described above, the abuses we’ve uncovered have informed updates to our preventative safety measures. We have also shared details of our findings, including indicators of misuse, with third-party safety teams.

In the full report, we address a number of other malicious uses of our models, including an attempt to compromise Vietnamese telecommunications infrastructure, and the use of multiple AI agents to commit fraud. The growth of AI-enhanced fraud and cybercrime is particularly concerning to us, and we plan to prioritize further research in this area.

We’re committed to continually improving our methods for detecting and mitigating these harmful uses of our models. We hope this report helps those in industry, government, and the wider research community strengthen their own defenses against the abuse of AI systems.

Further reading

For the full report with additional case studies, see here.



Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

AI Research

Canada’s first lunar rover looks to future space exploration

Published

on


Ali Abbas AhmadiBBC News, Toronto

Canadian Space Agency A computer generated image of the lunar vehicle on the surface of the MoonCanadian Space Agency

A computer generated image of what the lunar vehicle could look like on the Moon

In a shopping plaza an hour outside Toronto, flanked by a day spa and a shawarma joint, sits a two-storey building with blue tinted windows reflecting the summer sun.

It is the modest headquarters of Canadensys Aerospace, where Canada is charting its first trip to the Moon.

Canadensys is developing the first-ever Canadian-built rover for exploring the Earth’s only natural satellite, in what will be the first Canadian-led planetary exploration endeavour.

Models, maps and posters of outer space line the office walls, while engineers wearing anti-static coats work on unfamiliar-looking machines.

Sending this rover to the Moon is part of the company’s “broader strategy of really moving humanity off the Earth”, Dr Christian Sallaberger, Canadensys’ president and CEO, told the BBC.

Learning about the Moon – which is seen to have the potential to become a base for further space exploration – is the “logical first step”, he said.

“People get all excited about science fiction films when they come out. You know, Star Wars or Star Trek. This is the real thing.”

Two prototypes of the Moon rover, one white one in the foreground and the grey one on the table, with the Canadian flag in the background

Prototypes of the lunar rover, both designed and built by Canadensys

The Canadian vehicle is part of Nasa’s Artemis programme, which aims to establish a sustainable human presence on the Moon.

As part of that overarching goal, this rover aims to find water and measure radiation levels on the lunar surface in preparation for future manned missions, and survive multiple lunar nights (equivalent to about 14 days on Earth).

The rover will also demonstrate Canadian technology, building on Canada’s history in space.

Canada was the third country to launch a satellite, designed the Canadarm robotic arms for the Space Shuttle and the International Space Station, and is known for astronauts such as Chris Hadfield and Jeremy Hansen – the latter of whom will orbit the Moon on the Artemis II mission next year.

The 35kg rover is scheduled to be launched as part of a Nasa initiative in 2029 at the earliest. It will land on the Moon’s south polar region – one of the most inhospitable places on the lunar surface.

The vehicle does not have a name yet. The Canadian Space Agency held an online competition to select one, and is expected to announce the winner in the future.

Canadensys President Christian Sallaberger standing in one of the company's labs, with a Canadian flag visible on the back wall

Canadensys President Christian Sallaberger said he is excited to be playing a role in humanity’s quest to explore space

Canadensys is currently working on several prototypes of the rover. The final vehicle, Mr Sallaberger said, would be assembled shortly before launch.

Each component is tested to ensure it can survive the Moon’s harsh conditions.

Temperature is one of the main obstacles. Lunar nights can plummet to -200C (-328F) and rise to a scorching daytime of 100C (212F).

“It’s one of the biggest engineering challenges we have because it’s not so much even surviving the cold temperature, but swinging between very cold and very hot,” he said.

Designing the wheels is another challenge, as the Moon’s surface is covered with a sticky layer of fragmented rock and dust called regolith.

“Earth dirt, if you look at it microscopically, has been weathered off. It’s more or less in a round shape; but on the Moon the lunar dirt soil is all jagged,” Mr Sallaberger said.

“It’s like Velcro dirt,” he said, noting it “just gums up mechanisms”.

Engineers Misha Hartmann and Adam Abdulahad work on a prototype of the rover at the Canadensys headquarters. Both wear white lab coats and are working on the vehicle, which is siting on a table, and has metal wheels and visible wires.

Engineers Misha Hartmann (L) and Adam Abdulahad work on a prototype of the rover at the Canadensys headquarters

The search for water on the lunar surface is especially exciting, considering the Moon was generally thought to be bone dry following the Apollo missions in the 1960s and 70s, the US human spaceflight programme led by Nasa.

That perception changed in 2008, Dr Gordon Osinski, the mission’s chief scientist, told the BBC, when researchers re-analysed some Apollo mission samples and found particles of water.

Around the same time, space crafts observing the Moon detected its presence from orbit.

It has yet to be verified on the ground and many questions remain, the professor at Western University in London, Ontario, said.

“Is it like a patch of ice the size of this table? The size of a hockey rink? Most people think, like in the Arctic, it’s probably more like grains of ice mixed in with the soil,” he said.

Water on the Moon could have huge implications for more sustainable exploration. He noted one of the heaviest things they need to transport is often water, so having a potential supply there would open doors.

Water molecules can also be broken down to obtain hydrogen, which is used in rocket fuel. Mr Osinski described a future where the Moon could become a sort of petrol station for spacecrafts.

“It gets more in the realms of sci- fi,” he said.

Dr Gordon Osinski seated at a wooden desk in his office in Western University, wearing an Artemis programme t shirt with a rock from the Moon on the desk in front of him

Dr Osinski, an expert in lunar geology and has experience training astronauts in Canada’s Arctic, showed off a lunar rock during the BBC interview

Canada has wanted to build a lunar surface vehicle for decades, with talk of a Canadian-made spacecraft even in the early 2000s – but it was not until 2019 that concrete plans were announced.

Canadensys was awarded the C$4.7m ($3.4m; £2.5m) contract three years later.

Founded in 2013, Canadensys has worked on a variety of aerospace projects for organisations like Nasa and the Canadian Space Agency, as well as commercial clients.

More than 20 instruments built by the company have been used in a host of missions on the Moon.

But there are challenges ahead – as even landing on the Moon is no easy feat.

In March, a spacecraft by commercial US firm Intuitive Machines toppled over onto its side during landing, ending the mission prematurely.

Three months later, Japanese company iSpace’s Resilience lost touch with Earth during its landing, and eventually failed.

“That’s the nature of the business we’re in,” Mr Sallaberger said. “Things do go wrong, and we try to do the best we can to mitigate that.”

Intuitive Machines/The Planetary Society An instrument designed by Canadensys, on the Intuitive Machines' lander in space. Earth is seen below it against a black sky. Intuitive Machines/The Planetary Society

A picture of the Earth taken by a Canadensys-built camera was selected as the Best Space Exploration Image of 2024 by the Planetary Society

Space exploration has been a collaborative field over the years, with countries – even rivals, such as the United States and Russia – working together on the International Space Station.

But that might be changing, Mr Osinski said. As the prospect of a permanent presence on the Moon becomes more realistic, wider geopolitical questions have begun to swirl around the ownership of the satellite.

“There’s more talk around who owns the Moon and space resources,” Mr Osinski said.

In 2021, the US passed a law to protect the Apollo Moon landing site “because they had a concern that China could just go and grab the US flag, or take a piece of an Apollo lander”, he said.

But he had some encouraging words about the Artemis missions, which are “even way more international than the space station”.

The Artemis Accords, which is a set of ideals to promote sustainable and peaceful exploration of outer space, has been signed by more than 50 countries – including ones like Uruguay, Estonia and Rwanda, which are not traditionally seen as key space race nations.

Space is also becoming more accessible. Private companies like SpaceX and Blue Origin have taken an increasingly important role and are able to take anyone with the money and barely any training – like Amazon founder Jeff Bezos and pop star Katy Perry – into space for a few minutes.

But the Moon is the Holy Grail, as it opens up all sorts of possibilities.

Mr Sallaberger said that Canadensys is involved in longer-term projects, such as lunar greenhouses for food production.

Those still remain many years in the future, but the rover is a starting point.

“If you design something that can survive on the lunar surface long-term, you’re pretty bulletproof anywhere else in the solar system.”



Source link

Continue Reading

AI Research

Researchers make AI-powered tool to detect plant diseases

Published

on


A team of researchers at Maharshi Dayanand University (MDU), Rohtak, has developed an artificial intelligence (AI)-based tool capable of detecting diseases and nutrient deficiencies in bitter gourd leaves, potentially transforming the way farmers monitor crop health.

The study, recently published in the peer-reviewed journal ‘Current Plant Biology’ (Elsevier), highlights how AI-driven innovations can play a crucial role in real-time crop monitoring and precision farming.

The newly developed web-based application, named ‘AgriCure’, is powered by a layered augmentation-enhanced deep learning model. It allows farmers to diagnose crop health by simply uploading or capturing a photograph of a leaf using a smartphone.

“Unlike traditional methods, which are time-consuming and often require expert intervention, AgriCure instantly analyses the image to determine whether the plant is suffering from a disease or nutrient deficiency, and then offers corrective suggestions,” explained the researchers.

The collaborative research project was led by Dr Kamaldeep Joshi, Dr Rainu Nandal and Dr Yogesh Kumar, along with students Sumit Kumar and Varun Kumar from MDU’s University Institute of Engineering and Technology (UIET). It also involved Prof Narendra Tuteja from the International Centre for Genetic Engineering and Biotechnology (ICGEB), New Delhi and Prof Ritu Gill and Prof Sarvajeet Singh Gill from MDU’s Centre for Biotechnology.

MDU Vice-Chancellor, Prof Rajbir Singh, congratulated the research team on their achievement.

According to the researchers, AgriCure can detect major diseases such as downy mildew, leaf spot, and jassid infestation, as well as key nutrient deficiencies like nitrogen, potassium and magnesium.

“This represents a step towards sustainable agriculture, where AI empowers farmers with real-time decision-making tools,” said corresponding authors Prof Ritu Gill and Prof Sarvajeet Singh Gill. They added that the web-based platform can be integrated with mobile devices for direct use in the field.

The team believes that the technology’s core framework can be extended to other crops such as cereals, legumes, and fruits, creating opportunities for wider applications across Indian agriculture.

Looking ahead, they plan to integrate AgriCure with drones and Internet of Things (IoT) devices for large-scale monitoring, and to develop lighter versions of the model for full offline use on mobile phones.





Source link

Continue Reading

AI Research

Competition to introduce artificial intelligence (AI) is fierce not only in industrial areas but als..

Published

on


Competition to introduce AI to the diplomatic front lines of major countries The U.S. actively utilizes the State Department’s exclusive “State Chat” to brainstorm foreign policy. Canada uses it to analyze major countries’ policies

[Photo = Yonhap News]

Competition to introduce artificial intelligence (AI) is fierce not only in industrial areas but also in diplomacy, which is the front line of competition between countries. The U.S. State Department is increasing the work efficiency of diplomats through its own AI. Japan spends more than 600 billion won a year to detect false information. The move is aimed at preventing the possibility that fake information will be misused to establish national diplomatic strategies.

In the United States, the State Department has been operating its own AI ‘State Chat’ since last year. It is an interactive AI in the form of ‘Chat GPT’, similar to the method promoted by the Korean Ministry of Foreign Affairs. It provides functions such as summarizing internal business documents and professional analysis. E-mails used by diplomats are also drafted according to the format and even have the function of helping “brainstorming” in relation to foreign policy or strategy.

StateChat is dramatically reducing the amount of time State Department employees spend on mechanical tasks. According to State Department estimates, the total amount of time saved by all employees through their own AI amounts to 20,000 to 30,000 hours per week.

The State Department plans to continue expanding the use of StateChat. State Chat is also used for job training. This is due to the advantage of minimizing information that may be omitted during the handover process and enabling in-depth learning by providing data containing stories. State Chat will also be used to manage manpower. Information related to personnel management is also entered in State Chat.

[Photo = Yonhap News]
[Photo = Yonhap News]

Japan has been building a situation analysis system using AI since 2022. AI finally judges the situation by combining reports from local diplomats with external information such as foreign social network service (SNS) posts, reports from research institutes, and media reports. For example, if social media analysis detects residents’ disturbance in a specific area, AI warns of the risk of terrorism or riots.

From 2023, it is using AI to detect fake news that is mainly spread through SNS. It analyzes not only text but also various media types of content such as images, audio, and video. It is a method of measuring the consistency of information based on a large language model (LLM) and then determining whether it is false. In particular, Japan calculates and presents the social impact, such as the scale and influence of the fake news.

Japan believes that numerous fake news after the Fukushima nuclear power plant accident has undermined national trust and caused unnecessary diplomatic friction. Japan allocated about 66.2 billion yen (626.5 billion won) in the fiscal 2025 budget to the policy and technology sectors to respond to false information.

Canada introduced a ‘briefing note’ using Generative AI in 2022. A draft policy briefing document is created by analyzing and reviewing policy-related data of major countries. Finland operates a system that collects diplomatic documents through AI and summarizes them on its own, and even visualization functions are provided. The UK has introduced AI to consular services. Classify the services frequently requested by their citizens staying abroad to overseas missions and provide optimal answers.

Last year, France developed an AI tool that summarizes and analyzes diplomatic documents and external data and is using it to detect ‘reverse information (fake news or false information)’ overseas and to identify public opinion trends. The United Arab Emirates (UAE) has introduced an unmanned overseas mission model that provides consular services based on AI.



Source link

Continue Reading

Trending