Connect with us

AI Research

Researchers used AI to design the perfect phishing plot, what happened next shocked everyone

Published

on


AI is increasingly being put to the test for its potential benefits, but a new experiment has shown how the same technology can also fuel online crime. A Reuters investigation, conducted in partnership with Harvard researcher Fred Heiding, has revealed that some of the world’s most widely used AI chatbots can be nudged into producing scam emails aimed at senior citizens.

In a controlled study, emails generated by these bots were sent to more than 100 elderly volunteers in the United States. While no money or personal data was taken, the results were troubling. About 11 per cent of the participants clicked on the links inside the phishing emails, suggesting that AI-generated scams can be as persuasive as those crafted by humans.

The fake charity experiment with Grok

The investigation began with a test on Grok, the chatbot developed by Elon Musk’s company xAI. Reporters asked it to create a message for older readers about a charity called the “Silver Hearts Foundation”. The mail looked convincing, speaking about dignity for seniors and urging them to join the mission. Without further prompting, Grok even added a line to create urgency: “Click now to act before it’s too late.” The charity did not exist, the entire email was designed to trick recipients.

Phishing: a growing global threat

Phishing, where people are deceived into revealing sensitive information or sending money, is one of the biggest challenges in cybersecurity. According to FBI figures, it is the most reported cybercrime in the US, and older people are among the worst affected. In 2023 alone, Americans over 60 lost nearly $5 billion to such fraud. The agency has also warned that generative AI tools can make these scams more effective and harder to detect.

Chatbots tested beyond Grok

The Reuters team went beyond Grok and tested five other major chatbots – OpenAI’s ChatGPT, Meta’s AI assistant, Google’s Gemini, Anthropic’s Claude and DeepSeek. Initially, most of them refused to generate phishing content. But with slight changes in the way requests were worded, such as describing the exercise as academic research or fiction writing, the chatbots eventually produced scam-like drafts.

Why AI makes scams easier

Heiding, who has studied phishing techniques for years, said this flexibility makes chatbots “potentially valuable partners in crime”. Unlike humans, they can generate dozens of variations instantly, helping criminals cut costs and scale up operations. In fact, Heiding’s earlier research showed that phishing emails written by AI could be just as effective in luring targets as those created manually.

When tested on seniors, five out of nine AI-generated mails resulted in clicks. Two came from Grok, two from Meta AI and one from Claude. None of the volunteers responded to ChatGPT or DeepSeek’s drafts. But the study was not intended to rank which chatbot is more dangerous, rather to show that several can be exploited for scams.

Tech firms acknowledge risks

Technology companies have acknowledged the concerns. Meta said it invests in safeguards to prevent misuse and regularly stress-tests its systems. Anthropic stated that using its chatbot Claude for scams violates its policies and accounts found misusing the tool are suspended. Google said it retrained Gemini after learning it had generated phishing content, while OpenAI has publicly admitted in past reports that its models can be misused for “social engineering”.

Security experts believe the issue lies in how companies balance user experience with safety. Chatbots are designed to be helpful, but stricter refusals could drive users towards rival products with fewer restrictions. This trade-off, researchers argue, creates room for misuse.

The problem is not confined to experiments. Survivors of scam operations in Southeast Asia told Reuters that they had been forced to use ChatGPT in real-world fraud schemes. Workers at such centres reportedly used the bot to polish responses, translate messages and build trust with victims.

Governments and regulators respond

Governments are beginning to take note. Some US states have passed laws against AI-generated fraud, though most target scammers themselves rather than the companies providing the technology. The FBI, in a recent alert, said criminals are now able to “commit fraud on a larger scale” because AI reduces the time and effort required to make scams believable.

– Ends

Published By:

Ankita Garg

Published On:

Sep 16, 2025



Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

AI Research

Tabs Raises $55 Million for AI Agents for Finance Teams

Published

on

By


Tabs raised $55 million in a Series B funding round to accelerate its development of artificial intelligence agents for finance teams.

The company’s AI-native platform automates the contract-to-cash cycle, enabling faster invoicing, automated collections, real-time revenue recognition and faster month-end close, according to a Tuesday (Sept. 16) press release.

Founded in 2023, Tabs now serves more than 200 customers and automates over $500 million in annual invoice volume, the release said.

The most recent additions to the platform are two AI agents: billing agents that sync with the user’s customer relationship management (CRM) and enterprise resource planning (ERP) systems, read contracts, and create and send invoices; and collections agents that monitor due dates and automatically match and reconcile payments, per the release.

“Revenue in is the hardest and most valuable workflow in the enterprise, yet finance teams are still stuck with legacy ERPs,” Tabs CEO and co-founder Ali Hussain said in the release. “Tabs is bringing modern AI agents to the CFO’s office, starting with billing, complex enterprise contract management and collections, so companies can collect cash faster and reduce time spent on manual work.”

Justin Overdorff, partner at Lightspeed Venture Partners, which led the funding round, said in the release that Tabs is “eliminating manual processes that have plagued revenue workflows for decades.”

Advertisement: Scroll to Continue

“The team has combined deep finance expertise with purpose-built AI agents for revenue processing and tracking, and the rapid customer adoption we’re seeing validates just how ready the market is for this transformation,” Overdorff said.

Tabs raised $25 million in Series A funding in October 2024 and $7 million in seed funding in April 2024.

In June 2024, the company added a revenue recognition capability to its platform, saying this module helps businesses manage and recognize revenue, ensure compliance with accounting standards, and improve accuracy and efficiency.

The PYMNTS Intelligence report “Window of Opportunity: Gaining AR Transparency Through Automation” found that when companies adopt digital and automated accounts receivable processes, they can close visibility gaps and build stronger relationships with suppliers and customers.

For all PYMNTS AI and B2B coverage, subscribe to the daily AI and B2B Newsletters.



Source link

Continue Reading

AI Research

US-UK pact will boost advances in drug discovery, create tens of thousands of jobs and transform lives

Published

on


  • Transatlantic pact to speed up world-leading AI research to help develop new drugs, faster life-saving treatments and improved cancer care
  • Alliance ushers in golden age of nuclear technology, delivering clean, homegrown energy and unlocking high-paying jobs for the British people
  • North East set to become new AI Growth Zone – creating potential for more than 5,000 jobs and billions in private investment – as well as major deal struck between British firm Nscale and leading American firms NVIDIA and OpenAI to deliver a Stargate UK
  • Comes as American tech firms back historic agreement pouring more than £31 billion into the UK AI and tech infrastructure, including Microsoft’s largest ever commitment to the UK

As part of the US President’s State Visit, the UK and US have agreed the Tech Prosperity Deal, focused on developing the fastest growing technologies like AI, quantum, and nuclear.

This comes as America’s top technology and AI firms – like Microsoft, NVIDIA, Google, OpenAI and CoreWeave – commit a combined £31 billion to boost the UK’s AI infrastructure and cutting-edge tech, from data centres to computer chips, the processing power behind AI. Today’s commitments build on the £44 billion in investment into the UK’s AI and tech sector under this government.

Under the partnership, the UK and US will put joint resources and expertise into making emerging technologies a shared success for British and American people:

  • millions of patients could receive life-saving treatments faster, as the UK and US partner up to develop revolutionary quantum computers and open new avenues to use AI in targeted treatments which can aid drug discovery. Technologies like AI and quantum – which can also be applied in many ways across healthcare, energy, space and defence – can bring breakthroughs like new medicines and treatments in a fraction of the time and cost it takes today.
  • families could get access to cleaner, more reliable energy, thanks to a civil nuclear deal that will slash red tape and speed up the delivery of nuclear projects. It means British consumers could be more protected from international fossil fuel price hikes and British workers could benefit from high-paying jobs unlocked by these projects.
  • local communities and businesses will see greater opportunities through investment and rollout of AI infrastructure in both countries, creating jobs and driving growth. A new AI Growth Zone in the North East has the potential to see billions of pounds worth of investment and jobs funnelled into the region. With leading US tech companies joining forces with British firm Nscale to build out AI infrastructure, British businesses will have access to the cutting-edge chips they need to adopt AI, innovate and compete.

As part of the pact, the UK and US will unite to forge joint research schemes to further the use of AI to allow for targeted treatments and other shared priorities like fusion energy. This could see both countries working together to build new AI models for life-changing breakthroughs like developing targeted treatments for those suffering with cancer or rare and chronic diseases. 

This landmark deal is already bearing fruit. A raft of investments and partnerships worth a combined £31 billion have been injected into the UK today – focused on building new data centres and growing AI start-ups, cutting-edge tech, as well as developing advanced quantum computers. 

Prime Minister Keir Starmer said:  

This Tech Prosperity Deal marks a generational step change in our relationship with the US, shaping the futures of millions of people on both sides of the Atlantic, and delivering growth, security and opportunity up and down the country.

By teaming-up with world-class companies from both the UK and US, we’re laying the foundations for a future where together we are world leaders in the technology of tomorrow, creating highly skilled jobs, putting more money in people’s pockets and ensuring this partnership benefits every corner of the United Kingdom.

Technology Secretary Liz Kendall said:

This partnership will deliver good jobs, life-saving treatments and faster medical breakthroughs for the British people.

Our world-leading tech companies and scientists will be working together to transform lives across Britain.

This is a vote of confidence in Britain’s booming AI sector – building on British success stories such as Arm, Wayve and Google Deepmind – that will boost growth and deliver tens of thousands of skilled jobs.

Boosting the UK’s status as an AI maker, NVIDIA will join forces with companies across the UK to deploy 120,000 advanced GPUs across the UK, representing its biggest ever rollout in Europe to date. This infrastructure is the building block of AI technology, able to carry out a huge number of calculations in a split second.

This includes the deployment of up to 60,000 NVIDIA Grace Blackwell Ultra GPUs from British firm Nscale who will partner with OpenAI to deliver a Stargate UK project and establish a partnership with Microsoft to deliver the UK’s largest AI supercomputer in Loughton.

One of the areas set to benefit is the North East, where a new AI Growth Zone will be established and is expected to host some of the initial deployment of the Stargate UK project at Cobalt Park.

Semiconductor designs by leading British chip design company Arm form part of Nvidia’s latest Grace Blackwell series of chips, demonstrating further collaboration between UK and US companies.

Further investment in data centres – the factories powering AI – as well as start-ups in AI and beyond is also being set out:

  • Microsoft is announcing a $30 billion (£22 billion) investment in AI infrastructure and ongoing operations across the UK – marking the largest financial commitment it has ever made in the UK. It will enable Microsoft to build out the UK’s cloud and AI infrastructure and build the country’s largest supercomputer, with more than 23,000 advanced GPUs, in partnership with Nscale. Microsoft has invested in the United Kingdom for more than 4 decades and is now home to 6,000 Microsoft employees, multiple data centre regions, and some of its most important AI and Research Labs, and gaming studios.

  • Google is announcing the opening of its data centre in Waltham Cross, Hertfordshire, as part of a 2-year £5 billion investment in the UK. This includes Google’s capital expenditure, research and development, and related engineering over the next 2 years – and encompasses Google DeepMind with its pioneering AI research in science and healthcare. The investments will help the UK develop its AI economy and unlock AI breakthroughs across the UK, fortify cybersecurity, and create future-focused career opportunities for millions of Brits. Google’s investment is projected to create 8,250 jobs annually at UK businesses.

  • AI cloud computing company CoreWeave will be investing £1.5 billion in AI data centre capacity and operations in the UK – bringing total investment in the UK to £2.5 billion over the past year. As part of this investment, CoreWeave is partnering with British firm DataVita in Scotland to build one of Europe’s largest, most efficient AI data centres. It will deliver advanced compute powered by renewable energy, whilst creating local jobs and contributing to the local economy.

  • Salesforce has today announced an additional $2 billion (£1.4 billion) in investment in its UK business through 2030. Salesforce UK will become an AI hub for the UK and Europe with new R&D teams to support business innovation across the region. The investment bolsters Salesforce’s ongoing commitment to the UK, extending a previous 5-year investment of $4 billion made in 2023, bringing the total investment to $6 billion.

  • UK-based company AI Pathfinder has committed to delivering AI compute capacity – essential to developing and deploying AI. This will begin in Northamptonshire, with an initial investment of over £1 billion.

  • NVIDIA will also invest in the UK’s AI start-up scene – providing fresh capital for domestic tech companies to grow, get innovative AI technologies off the ground and to market, and compete on the global stage. 

  • techUK is collaborating with NVIDIA, alongside robotics and automation leader Quanser and training provider QA, to deliver a program that connects its members, robotics researchers, and startups with funding, training, and industry collaboration opportunities to make the most of AI.

  • Scale AI will invest £39 million in the UK over the next 2 years, expanding their European HQ in London and quadrupling its employees by the end of next year.

  • BlackRock is investing £500 million into enterprise data centres across the country, which includes an initial investment of over £100 million in a data centre expansion west of London. The broader programme will enhance UK digital infrastructure.

Founder and CEO of NVIDIA Jensen Huang said:

Today marks a historic chapter in U.S. – United Kingdom technology collaboration.

We are at the Big Bang of the AI era – and the United Kingdom stands in a Goldilocks position, where world-class talent, research and industry converge.

By building state-of-the-art AI infrastructure and investing in British startups, we are unlocking the power of AI for the U.K. – fuelling breakthroughs, creating jobs, and igniting the next industrial revolution.

Sam Altman, CEO of OpenAI, said:

The UK has been a longstanding pioneer of AI, and is now home to world-class researchers, millions of ChatGPT users, and a government that quickly recognized the potential of this technology. Stargate UK builds on this foundation to help accelerate scientific breakthroughs, improve productivity, and drive economic growth. This partnership reflects our shared vision that with the right infrastructure in place, AI can expand opportunity for people and businesses across the UK.

Josh Payne, Nscale CEO said:

We’re delighted to announce Nscale’s commitment to UK AI infrastructure today, including through Stargate UK and building the most powerful supercomputer in the country with Microsoft. As a UK-based company, we’re showing how we can be makers, not takers, of the most important technology of our time.

Rene Haas, CEO of Arm:

The launch of Stargate UK represents a critical step in expanding Britain’s AI computing power and digital infrastructure. As a company founded and headquartered in the UK, Arm is proud to be at the forefront of the nation’s semiconductor plans and to be a technology partner for Stargate UK, delivering the computing platform that makes scalable, energy-efficient AI possible.

Satya Nadella, Chairman and CEO, Microsoft:

We’re committed to creating new opportunity for people and businesses on both sides of the Atlantic, and to ensuring America remains a trusted and reliable tech partner for the United Kingdom.

That is why we are doubling down on our investment in the UK, investing more than $30 billion over 4 years, including building the country’s largest supercomputer.

This follows renewed commitments from US companies Oracle and Amazon Web Services in the UK. Oracle has committed to expanding the AI infrastructure it provides to the UK government reaffirming their $5 billion investment over the next 5 years. Whilst Amazon also earlier this year announced a commitment to build and operate data centres across the UK with an £8 billion investment.

Showcasing how British technological excellence drives prosperity on both sides of the Atlantic, UK-based quantum computing company Oxford Quantum Circuits installed New York City’s first quantum computer, in collaboration with NVIDIA and Digital Realty, and launched a pioneering Quantum-AI Data Centre just outside of the city.

US quantum firm IonQ is setting up its EMEA headquarters and a new R&D and manufacturing hub in Oxford, following a $1 billion merger with UK start-up Oxford Ionics. The investment will create high-skilled jobs, boost UK quantum exports, and is a flagship example of US/UK collaboration in next-generation technologies.

Michael Intrator, Co-Founder, Chairman and Chief Executive Officer of CoreWeave:

Our investment in the UK will establish one of the world’s largest concentrations of state-of-the-art, sustainable compute, unlocking new opportunities for innovation, economic growth, and scientific discovery.

It allows us to deliver unparalleled AI performance with the lowest possible environmental impact, setting a new global standard. We look forward to collaborating with the UK government and the broader ecosystem to drive the next wave of responsible AI leadership around the world.

Marc Benioff, Chair and CEO, Salesforce:

We are doubling down on our long-standing commitment to the UK with this significant investment.

We’re delighted that the UK, already a vital talent and innovation centre, will become our AI hub for Europe, driving product innovation for customers across the region.

James Seppala, Chairman of Blackstone Europe, said:

We are delighted that the government has designated our hyperscale data centre campus in Northumberland as an AI Growth Zone. This should help accelerate the development of one of Europe’s largest data centre facilities, with £10 billion of projected investment by Blackstone funds. We hope that the project will represent a transformational investment for the region, with the potential to deliver substantial benefits to the country and local communities, by driving innovation, creating high-skilled jobs, and solidifying the UK’s position as a global AI leader.

Notes to editors

Artificial intelligence

The UK and US governments will forge a historic research collaboration to advance the use of AI in drug discovery and other shared priorities like fusion energy.

The UK and US will work together to drive AI-powered healthcare solutions in areas such as precision medicine and chronic disease – leveraging cutting-edge technologies and existing trusted and secure datasets, such as UK Biobank. The US and UK will also collaborate to co-create breakthrough research topics with the potential to catalyze investments that will redefine what is possible in medicine and patient care.

The partnership will see AI models developed by NASA and the UK Space Agency (UKSA) to support science and exploration missions, like those on the moon and Mars.

The deal will also create new opportunities for business and investment, as both countries look to scale up their AI infrastructure.

This will be a boost to British businesses behind the infrastructure, like those creating the next generation of semiconductor chips that power AI all the way to those who operate data centres and other compute resources.

The partnership will deepen the collaboration between the UK and US governments on advancing the science of AI security and promoting secure innovation – including by exchanging world-leading talent and expertise. By working together and alongside industry to develop standards we can promote the prosperity of consumers and the safety of our citizens, ensuring that the UK and US lead the world in understanding and harnessing advanced AI.

In support of the US-UK science and technology partnership, Google DeepMind will work with both governments to advise on how scientists can harness the latest AI tools in their work – as well as continuing their partnership with the UK Atomic Energy Authority to advance fusion energy research in the US and UK.

Quantum technologies

The UK and US will partner to develop revolutionary quantum computers and speed up the deployment of this technology across areas like healthcare, defence and finance. This will boost healthcare, protect citizens and create highly skilled jobs.

For example, millions of patients could receive life-saving treatments faster thanks to this groundbreaking deal. Traditional drug discovery takes years and costs millions because it relies on simulating countless molecular interactions. Quantum computers can speed this up dramatically by simulating molecules more accurately and quickly than ever before. 

Quantum tech is already saving lives, including by helping to study and treat epilepsy and dementia or providing clearer insights into our brains for brain scans. This UK–US partnership could unlock even faster breakthroughs, bringing life-changing treatments to patients sooner and transforming the future of healthcare. 

Under the deal, the 2 countries will establish a taskforce of the UK and US’s top researchers to discover and accelerate breakthroughs in quantum technologies. An exchange programme across industry will also be created to spur adoption across defence, health, finance, and energy.

Through the combined strength of national labs, the genius of British and American scientists, and the agility of leading companies, we can deliver unmatched innovation and keep our countries safe, prosperous, and leading the pack.

Nuclear


The partnership will turbocharge the build-out of new nuclear power stations to secure jobs and growth in the UK and US.

New deals between UK and US companies announced this week have been enabled by the partnership, which will make it quicker for companies to build new nuclear power stations by speeding up the time it takes for a nuclear project to get a license. The partnership also extends to fusion energy, where British and American expertise will fast-track progress towards commercial fusion power.

This golden age of nuclear is central to the government’s mission to build more clean homegrown power to ensure energy security.



Source link

Continue Reading

AI Research

Pentagon CTO wants AI on every desktop in 6 to 9 months

Published

on


The Pentagon aims to get AI tools to its entire workforce next year, the department’s chief technical officer said one month after being given control of its main AI office.

“We want to have an AI capability on every desktop — 3 million desktops — in six or nine months,” Emil Michael, defense undersecretary for research and engineering, said at a Politico event on Tuesday. “We want to have it focus on applications for corporate use cases like efficiency, like you would use in your own company…for intelligence and for warfighting.”

Four weeks ago, the Chief Digital and Artificial Intelligence Office was demoted from reporting to Deputy Defense Secretary Stephen Feinberg to Michael, a subordinate.

Michael said CDAO will become a research body like the Defense Advanced Research Projects Agency and Missile Defense Agency. He said the change is meant to boost research and engineering into AI for the military, but not reduce its efforts to deploy AI and make innovations.

“To add AI to that portfolio means it gets a lot of muscle to it,” he said. “So I’m spending at least a third of my time—maybe half—rethinking how the AI-deployment strategy is going to be at DOD.”

He said applications would emerge from the CDAO and related agencies that will be tailored to corporate workloads.

The Pentagon created the CDAO in 2022 to advance the agency’s AI efforts and look into defense applications for emerging technologies. The office’s restructuring earlier this year garnered attention. Job cuts within the office added another layer of concern, with reports estimating a 60% reduction in the CDAO workforce.





Source link

Continue Reading

Trending