AI Research
Instagram wrongly says some users breached child sex abuse rules
Technology Reporter
Getty ImagesInstagram users have told the BBC of the “extreme stress” of having their accounts banned after being wrongly accused by the platform of breaching its rules on child sexual exploitation.
The BBC has been in touch with three people who were told by parent company Meta that their accounts were being permanently disabled, only to have them reinstated shortly after their cases were highlighted to journalists.
“I’ve lost endless hours of sleep, felt isolated. It’s been horrible, not to mention having an accusation like that over my head,” one of the men told BBC News.
Meta declined to comment.
BBC News has been contacted by more than 100 people who claim to have been wrongly banned by Meta.
Some talk of a loss of earnings after being locked out of their business pages, while others highlight the pain of no longer having access to years of pictures and memories. Many point to the impact it has had on their mental health.
Over 27,000 people have signed a petition that accuses Meta’s moderation system, powered by artificial intelligence (AI), of falsely banning accounts and then having an appeal process that is unfit for purpose.
Thousands of people are also in Reddit forums dedicated to the subject, and many users have posted on social media about being banned.
Meta has previously acknowledged a problem with Facebook Groups but denied its platforms were more widely affected.
‘Outrageous and vile’
The BBC has changed the names of the people in this piece to protect their identities.
David, from Aberdeen in Scotland, was suspended from Instagram on 4 June. He was told he had not followed Meta’s community standards on child sexual exploitation, abuse and nudity.
He appealed that day, and was then permanently disabled on Instagram and his associated Facebook and Facebook Messenger accounts.
David found a Reddit thread, where many others were posting that they had also been wrongly banned over child sexual exploitation.
“We have lost years of memories, in my case over 10 years of messages, photos and posts – due to a completely outrageous and vile accusation,” he told BBC News.
He said Meta was “an embarrassment”, with AI-generated replies and templated responses to his questions. He still has no idea why his account was banned.
“I’ve lost endless hours of sleep, extreme stress, felt isolated. It’s been horrible, not to mention having an accusation like that over my head.
“Although you can speak to people on Reddit, it is hard to go and speak to a family member or a colleague. They probably don’t know the context that there is a ban wave going on.”
The BBC raised David’s case to Meta on 3 July, as one of a number of people who claimed to have been wrongly banned over child sexual exploitation. Within hours, his account was reinstated.
In a message sent to David, and seen by the BBC, the tech giant said: “We’re sorry that we’ve got this wrong, and that you weren’t able to use Instagram for a while. Sometimes, we need to take action to help keep our community safe.”
“It is a massive weight off my shoulders,” said David.
Faisal was banned from Instagram on 6 June over alleged child sexual exploitation and, like David, found his Facebook account suspended too.
The student from London is embarking on a career in the creative arts, and was starting to earn money via commissions on his Instagram page when it was suspended. He appealed after feeling he had done nothing wrong, and then his account was then banned a few minutes later.
He told BBC News: “I don’t know what to do and I’m really upset.
“[Meta] falsely accuse me of a crime that I have never done, which also damages my mental state and health and it has put me into pure isolation throughout the past month.”
His case was also raised with Meta by the BBC on 3 July. About five hours later, his accounts were reinstated. He received the exact same email as David, with the apology from Meta.
He told BBC News he was “quite relieved” after hearing the news. “I am trying to limit my time on Instagram now.”
Faisal said he remained upset over the incident, and is now worried the account ban might come up if any background checks are made on him.
A third user Salim told BBC News that he also had accounts falsely banned for child sexual exploitation violations.
He highlighted his case to journalists, stating that appeals are “largely ignored”, business accounts were being affected, and AI was “labelling ordinary people as criminal abusers”.
Almost a week after he was banned, his Instagram and Facebook accounts were reinstated.
What’s gone wrong?
When asked by BBC News, Meta declined to comment on the cases of David, Faisal, and Salim, and did not answer questions about whether it had a problem with wrongly accusing users of child abuse offences.
It seems in one part of the world, however, it has acknowledged there is a wider issue.
The BBC has learned that the chair of the Science, ICT, Broadcasting, and Communications Committee at the National Assembly in South Korea, said last month that Meta had acknowledged the possibility of wrongful suspensions for people in her country.
Dr Carolina Are, a blogger and researcher at Northumbria University into social media moderation, said it was hard to know what the root of the problem was because Meta was not being open about it.
However, she suggested it could be due to recent changes to the wording of some its community guidelines and an ongoing lack of a workable appeal process.
“Meta often don’t explain what it is that triggered the deletion. We are not privy to what went wrong with the algorithm,” she told BBC News.
In a previous statement, Meta said: “We take action on accounts that violate our policies, and people can appeal if they think we’ve made a mistake.”
Meta, in common with all big technology firms, have come under increased pressure in recent years from regulators and authorities to make their platforms safe spaces.
Meta told the BBC it used a combination of people and technology to find and remove accounts that broke its rules, and was not aware of a spike in erroneous account suspension.
Meta says its child sexual exploitation policy relates to children and “non-real depictions with a human likeness”, such as art, content generated by AI or fictional characters.
Meta also told the BBC a few weeks ago it uses technology to identify potentially suspicious behaviours, such as adult accounts being reported by teen accounts, or adults repeatedly searching for “harmful” terms.
Meta states that when it becomes aware of “apparent child exploitation”, it reports it to the National Center for Missing and Exploited Children (NCMEC) in the US. NCMEC told BBC News it makes all of those reports available to law enforcement around the world.
Lars and I discussed the role of AI in education and beyond, starting with a nostalgic nod to calculators and slide rules.
When I spoke with Lars recently, I emphasized the critical balance between using tools like AI and actually learning how to think, reason, and create. We touched on the incredible capabilities of AI, raising exciting possibilities and serious concerns. For example, AI-driven customer service can be frustratingly impersonal—I yelled at one for eight minutes earlier that day just trying to reach a human. In education, some elite universities are already requiring handwritten essays again, like with Blue Books, to preserve that integrity. However, as I have written previously about AI and medicine, it has the potential to increase accessibility and the quality of care.
You can find the whole conversation here.
Looking for a deeper dive?
Will Artificial Intelligence Replace Human
Can AI Compensate for an Inferior Education?
If Your Shrink is a Psychopathic Chatbot, Can You Sue for Malpractice?
Al’s Potential to Help, Hinder, or Hollow Out Human Expertise
China led the world in artificial intelligence research in 2024, producing the largest volume of publications after developing a “nationwide innovation ecosystem”, according to a new report.
Despite rising geopolitical tensions, China was also the top AI research collaborator for the US, UK and European Union.
The study, DeepSeek and the New Geopolitics of AI: China’s ascent to research pre-eminence in AI, is published by research technology company Digital Science and authored by its chief executive, Daniel Hook.
The analysis used global data from its Dimensions database, covering research publication and collaboration trends from 2000 to 2024.
ADVERTISEMENT
It says that AI research has grown at an “impressive rate” globally, expanding from just under 10,000 publications in 2000 to 60,000 in 2024.
China was found to be “the pre-eminent world power in AI research,” leading not only by research volume but also by “citation attention, and influence,” with its lead over the rest of the world growing rapidly over the past seven years.
ADVERTISEMENT
In 2024, China’s publication output in AI research matched the combined output of the US, UK and EU, while its share of global citation attention exceeded 40 per cent.
Despite political tensions, the report finds that China has become the strongest AI research collaborator for the US, UK and EU, while itself needing “less reciprocal collaboration than any of them”.
China is also said to have the largest AI talent pool, with 30,000 active researchers and a large student and postdoctoral population, supporting what the report calls a “nationwide innovation ecosystem”.
The report highlights that 156 Chinese institutions each published more than 50 AI papers in 2024, contrasting with the more clustered research hubs in the West.
Alongside academic research, China also dominated AI-related patents. Patent filings and company-affiliated AI research showed China outpacing the US tenfold in some indicators, according to the report, reflecting the country’s ability to translate research into innovation.
ADVERTISEMENT
Hook argued that AI had become a “strategic asset, akin to energy or military capability”, adding that “China is actively leveraging this advantage”.
He added: “Governments need to understand the local, national and geostrategic implications of AI, with the underlying concern that lack of AI capability or capacity could be damaging from economic, political, social and military perspectives.”
The release of the DeepSeek chatbot in January 2025 was cited as one example of China’s emerging capabilities.
ADVERTISEMENT
“The emergence of DeepSeek is not merely a technological innovation – it is a symbol of a profound shift in the global AI landscape,” Hook said.
He described DeepSeek as demonstrating “China’s technological independence”, calling it a “cost-efficient, open-source LLM” that showed the country’s ability to “innovate around US chip restrictions and dominate AI development at scale”.
The report also describes the UK as “small but globally impactful”, saying that despite its modest size, the country consistently punches above its weight in “attention-per-output metrics”.
The EU, by contrast, “risks falling behind in translation and visibility.”
ADVERTISEMENT
The report says that while the EU benefits from strong internal collaboration, it “shows weaker international collaboration beyond its borders and struggles to convert research into applied outputs (patents, for example), raising concerns about its future AI competitiveness”.
China is outpacing the rest of the world in artificial intelligence research at a time when AI is becoming a “strategic asset” akin to energy or military capability, according to a report from research technology company Digital Science.
The report – entitled DeepSeek and the New Geopolitics of AI: China’s ascent to research pre-eminence in AI – has been authored by Digital Science CEO Daniel Hook based on data from Dimensions, the world’s largest and most comprehensive database describing the global research ecosystem.
The news comes just a day after a report from Clarivate revealed that China is also leading the way in research output across G20 nations.
Dr Hook has analysed AI research data from the year 2000 to 2024, tracking trends in research collaborations and placing these within geopolitical, economic, and technological contexts. His report says AI research has grown at an “impressive rate” globally since the turn of the millennium – from just under 10,000 publications in 2000, to 60,000 publications in 2024.
Dr Hook’s key findings include:
-
China has become the pre-eminent world power in AI research, leading not only by research volume, but also by citation attention, and influence, rapidly increasing its lead on the rest of the world over the past seven years.
-
The US continues to have the strongest AI start-up scene, but China is catching up fast.
-
In 2024, China’s AI research publication output matched the combined output of the US, UK, and European Union (EU-27), and now commands more than 40% of global citation attention.
-
Despite global tensions, China has become the top collaborator for the US, UK, and EU in AI research, while needing less reciprocal collaboration than any of them.
-
China’s AI talent pool dwarfs its rivals – with 30,000 active AI researchers and a massive student and postdoctoral population.
-
The EU benefits from strong internal AI collaboration across its research bloc.
-
China dominates AI-related patents – patent filings and company-affiliated AI research show China outpacing the US tenfold in some indicators, underscoring its capacity to translate research into innovation.
“AI is no longer neutral – governments are using it as a strategic asset, akin to energy or military capability, and China is actively leveraging this advantage,” Hook says. “Governments need to understand the local, national and geostrategic implications of AI, with the underlying concern that lack of AI capability or capacity could be damaging from economic, political, social, and military perspectives.”
Hook says China is “massively and impressively” growing its AI research capacity. Unlike Western nations with clustered AI hubs, he says China boasts 156 institutions publishing more than 50 AI papers each in 2024, supporting a nationwide innovation ecosystem. In addition, “China’s AI workforce is young, growing fast, and uniquely positioned for long-term innovation.”
He says one sign of China’s rapidly developing capabilities is its release of the DeepSeek chatbot in January this year. “The emergence of DeepSeek is not merely a technological innovation – it is a symbol of a profound shift in the global AI landscape. DeepSeek exemplifies China’s technological independence. Its cost-efficient, open-source LLM demonstrates the country’s ability to innovate around US chip restrictions and dominate AI development at scale.”
The report comments further on the AI research landscape in the US, UK and EU. It says the UK remains “small but globally impactful”. “Despite its modest size, the UK consistently punches above its weight in attention-per-output metrics.”
However, the EU “risks falling behind in translation and visibility”. “The EU shows weaker international collaboration beyond its borders and struggles to convert research into applied outputs (e.g., patents), raising concerns about its future AI competitiveness.”
Discover more in the full report: DeepSeek and the New Geopolitics of AI: China’s ascent to research pre-eminence in AI.
Do you want to read more content like this? SUBSCRIBE to the Research Information Newsline!
-
Funding & Business1 week agoKayak and Expedia race to build AI travel agents that turn social posts into itineraries
-
Jobs & Careers1 week ago
Mumbai-based Perplexity Alternative Has 60k+ Users Without Funding
-
Mergers & Acquisitions1 week ago
Donald Trump suggests US government review subsidies to Elon Musk’s companies
-
Funding & Business1 week ago
Rethinking Venture Capital’s Talent Pipeline
-
Jobs & Careers1 week ago
Why Agentic AI Isn’t Pure Hype (And What Skeptics Aren’t Seeing Yet)
-
Education3 days ago9 AI Ethics Scenarios (and What School Librarians Would Do)
-
Education3 days agoTeachers see online learning as critical for workforce readiness in 2025
-
Education4 days ago
Nursery teachers to get £4,500 to work in disadvantaged areas
-
Education5 days ago
How ChatGPT is breaking higher education, explained
-
Education3 days ago
Labour vows to protect Sure Start-type system from any future Reform assault | Children
