AI is increasingly being put to the test for its potential benefits, but a new experiment has shown how the same technology can also fuel online crime. A Reuters investigation, conducted in partnership with Harvard researcher Fred Heiding, has revealed that some of the world’s most widely used AI chatbots can be nudged into producing scam emails aimed at senior citizens.

In a controlled study, emails generated by these bots were sent to more than 100 elderly volunteers in the United States. While no money or personal data was taken, the results were troubling. About 11 per cent of the participants clicked on the links inside the phishing emails, suggesting that AI-generated scams can be as persuasive as those crafted by humans.

The fake charity experiment with Grok

The investigation began with a test on Grok, the chatbot developed by Elon Musk’s company xAI. Reporters asked it to create a message for older readers about a charity called the “Silver Hearts Foundation”. The mail looked convincing, speaking about dignity for seniors and urging them to join the mission. Without further prompting, Grok even added a line to create urgency: “Click now to act before it’s too late.” The charity did not exist, the entire email was designed to trick recipients.

Phishing: a growing global threat

Phishing, where people are deceived into revealing sensitive information or sending money, is one of the biggest challenges in cybersecurity. According to FBI figures, it is the most reported cybercrime in the US, and older people are among the worst affected. In 2023 alone, Americans over 60 lost nearly $5 billion to such fraud. The agency has also warned that generative AI tools can make these scams more effective and harder to detect.

Chatbots tested beyond Grok

The Reuters team went beyond Grok and tested five other major chatbots – OpenAI’s ChatGPT, Meta’s AI assistant, Google’s Gemini, Anthropic’s Claude and DeepSeek. Initially, most of them refused to generate phishing content. But with slight changes in the way requests were worded, such as describing the exercise as academic research or fiction writing, the chatbots eventually produced scam-like drafts.

Why AI makes scams easier

Heiding, who has studied phishing techniques for years, said this flexibility makes chatbots “potentially valuable partners in crime”. Unlike humans, they can generate dozens of variations instantly, helping criminals cut costs and scale up operations. In fact, Heiding’s earlier research showed that phishing emails written by AI could be just as effective in luring targets as those created manually.

When tested on seniors, five out of nine AI-generated mails resulted in clicks. Two came from Grok, two from Meta AI and one from Claude. None of the volunteers responded to ChatGPT or DeepSeek’s drafts. But the study was not intended to rank which chatbot is more dangerous, rather to show that several can be exploited for scams.

Tech firms acknowledge risks

Technology companies have acknowledged the concerns. Meta said it invests in safeguards to prevent misuse and regularly stress-tests its systems. Anthropic stated that using its chatbot Claude for scams violates its policies and accounts found misusing the tool are suspended. Google said it retrained Gemini after learning it had generated phishing content, while OpenAI has publicly admitted in past reports that its models can be misused for “social engineering”.

Security experts believe the issue lies in how companies balance user experience with safety. Chatbots are designed to be helpful, but stricter refusals could drive users towards rival products with fewer restrictions. This trade-off, researchers argue, creates room for misuse.

The problem is not confined to experiments. Survivors of scam operations in Southeast Asia told Reuters that they had been forced to use ChatGPT in real-world fraud schemes. Workers at such centres reportedly used the bot to polish responses, translate messages and build trust with victims.

Governments and regulators respond

Governments are beginning to take note. Some US states have passed laws against AI-generated fraud, though most target scammers themselves rather than the companies providing the technology. The FBI, in a recent alert, said criminals are now able to “commit fraud on a larger scale” because AI reduces the time and effort required to make scams believable.

– Ends

SEERai Recognized as the Industry’s First Agentic AI Platform Transforming Cost, Schedule, and Risk Planning in Secure Enterprise Environments

LONG BEACH, Calif., Sept. 16, 2025 /PRNewswire/ — Galorath, the premier AI-powered operational intelligence platform provider, today announced that SEERai™ has been named a winner in SiliconANGLE’s 2025 TechForward Awards. The platform was recognized in the “AI Tech – Generative AI & Foundation Models” category for its impact in enabling secure, explainable AI-driven planning across complex programs.

SEERai is the first commercially available agentic AI platform engineered for program-critical outcomes. Unlike generic AI copilots or disconnected estimation tools, SEERai uses a modular architecture of purpose-built agents, retrieval-augmented generation (RAG), and structured decision logic to deliver fully traceable outputs. It enables organizations to accelerate proposal timelines, standardize estimation practices, and scale expert insight—without compromising accuracy, auditability, or security.