AI Research
Millions of websites to get ‘game-changing’ AI bot blocker
Senior Technology Reporter
Getty ImagesMillions of websites – including Sky News, The Associated Press and Buzzfeed – will now be able to block artificial intelligence (AI) bots from accessing their content without permission.
The new system is being rolled out by internet infrastructure firm, Cloudflare, which hosts around a fifth of the internet.
Eventually, sites will be able to ask for payment from AI firms in return for having their content scraped.
Many prominent writers, artists, musicians and actors have accused AI firms of training systems on their work without permission or payment.
In the UK, it led to a furious row between the government and artists including Sir Elton John over how to protect copyright.
Cloudflare’s tech targets AI firm bots – also known as crawlers – which are programs that explore the web, indexing and collecting data as they go. They are important to the way AI firms build, train and operate their systems.
So far, Cloudflare says its tech is active on a million websites.
Roger Lynch, chief executive of Condé Nast, whose print titles include GQ, Vogue, and The New Yorker, said the move was “a game-changer” for publishers.
“This is a critical step toward creating a fair value exchange on the Internet that protects creators, supports quality journalism and holds AI companies accountable”, he wrote in a statement.
However, other experts say stronger legal protections will still be needed.
‘Surviving the age of AI’
Initially the system will apply by default to new users of Cloudflare services, plus sites that participated in an earlier effort to block crawlers.
Many publishers accuse AI firms of using their content without permission.
Recently the BBC threatened to take legal action against US based AI firm Perplexity, demanding it immediately stopped using BBC content, and paid compensation for material already used.
However publishers are generally happy to allow crawlers from search engines, like Google, to access their sites, so that the search companies can in return can direct people to their content.
Perplexity accused the BBC of seeking to preserve “Google’s monopoly”.
But Cloudflare argues AI breaks the unwritten agreement between publishers and crawlers. AI crawlers, it argues, collect content like text, articles, and images to generate answers, without sending visitors to the original source—depriving content creators of revenue.
“If the Internet is going to survive the age of AI, we need to give publishers the control they deserve and build a new economic model that works for everyone,” wrote the firm’s chief executive Matthew Prince.
To that end the company is developing a “Pay Per Crawl” system, which would give content creators the option to request payment from AI companies for utilising their original content.
Battle the bots
According to Cloudflare there has been an explosion of AI bot activity.
“AI Crawlers generate more than 50 billion requests to the Cloudflare network every day”, the company wrote in March.
And there is growing concern that some AI crawlers are disregarding existing protocols for excluding bots.
In an effort to counter the worst offenders Cloudflare previously developed a system where the worst miscreants would be sent to a “Labyrinth” of web pages filled with AI generated junk.
The new system attempts to use technology to protect the content of websites and to give sites the option to charge AI firms a fee to access it.
In the UK there is an intense legislative battle between government, creators and the AI firms over the extent to which the creative industries should be protected from AI firms using their works to train systems without permission or payment.
And, on both sides of the Atlantic, content creators, licensors and owners have gone to court in an effort to prevent what they see as AI firms encroachment on creative rights.
Ed Newton-Rex, the founder of Fairly Trained which certifies that AI companies have trained their systems on properly licensed data, said it was a welcome development – but there was “only so much” one company could do
“This is really only a sticking plaster when what’s required is major surgery,” he told the BBC.
“It will only offer protection for people on websites they control – it’s like having body armour that stops working when you leave your house,” he added.
“The only real way to protect people’s content from theft by AI companies is through the law.”
Filmmaker Baroness Beeban Kidron, who is campaigning for more protection for the creative industries, welcomed the news saying the company had shown leadership.
“Cloudflare sits at the heart of the digital world and it is exciting to see them take decisive action,” she told the BBC.
“If we want a vibrant public sphere we need AI companies to contribute to the communities in which they operate, that means paying their fair share of tax, settling with those whose work they have stolen to build their products, and, as Cloudflare has just shown, using tech creatively to ensure equity between digital and human creators on an ongoing basis.”
AI Research
Radiomics-Based Artificial Intelligence and Machine Learning Approach for the Diagnosis and Prognosis of Idiopathic Pulmonary Fibrosis: A Systematic Review – Cureus
Artificial intelligence may take over some tasks and transform others, but one thing is certain: it’s reshaping the job market. Researchers at USC’s Information Sciences Institute (ISI) analyzed LinkedIn job postings and AI-related patent filings to measure which jobs are most exposed, and where those changes are happening first.
The project was led by ISI research assistant Eun Cheol Choi, working with students in a graduate-level USC Annenberg data science course taught by USC Viterbi Research Assistant Professor Luca Luceri. The team developed an “AI exposure” score to measure how closely each role is tied to current AI technologies. A high score suggests the job may be affected by automation, new tools, or shifts in how the work is done.
Which Industries Are Most Exposed to AI?
To understand how exposure shifted with new waves of innovation, the researchers compared patent data from before and after a major turning point. “We split the patent dataset into two parts, pre- and post-ChatGPT release, to see how job exposure scores changed in relation to fresh innovations,” Choi said. Released in late 2022, ChatGPT triggered a surge in generative AI development, investment, and patent filings.
Jobs in wholesale trade, transportation and warehousing, information, and manufacturing topped the list in both periods. Retail also showed high exposure early on, while healthcare and social assistance rose sharply after ChatGPT, likely due to new AI tools aimed at diagnostics, medical records, and clinical decision-making.
In contrast, education and real estate consistently showed low exposure, suggesting they are, at least for now, less likely to be reshaped by current AI technologies.
AI’s Reach Depends on the Role
AI exposure doesn’t just vary by industry, it also depends on the specific type of work. Jobs like software engineer and data scientist scored highest, since they involve building or deploying AI systems. Roles in manufacturing and repair, such as maintenance technician, also showed elevated exposure due to increased use of AI in automation and diagnostics.
At the other end of the spectrum, jobs like tax accountant, HR coordinator, and paralegal showed low exposure. They center on work that’s harder for AI to automate: nuanced reasoning, domain expertise, or dealing with people.
AI Exposure and Salary Don’t Always Move Together
The study also examined how AI exposure relates to pay. In general, jobs with higher exposure to current AI technologies were associated with higher salaries, likely reflecting the demand for new AI skills. That trend was strongest in the information sector, where software and data-related roles were both highly exposed and well compensated.
But in sectors like wholesale trade and transportation and warehousing, the opposite was true. Jobs with higher exposure in these industries tended to offer lower salaries, especially at the highest exposure levels. The researchers suggest this may signal the early effects of automation, where AI is starting to replace workers instead of augmenting them.
“In some industries, there may be synergy between workers and AI,” said Choi. “In others, it may point to competition or replacement.”
From Class Project to Ongoing Research
The contrast between industries where AI complements workers and those where it may replace them is something the team plans to investigate further. They hope to build on their framework by distinguishing between different types of impact — automation versus augmentation — and by tracking the emergence of new job categories driven by AI. “This kind of framework is exciting,” said Choi, “because it lets us capture those signals in real time.”
Luceri emphasized the value of hands-on research in the classroom: “It’s important to give students the chance to work on relevant and impactful problems where they can apply the theoretical tools they’ve learned to real-world data and questions,” he said. The paper, Mapping Labor Market Vulnerability in the Age of AI: Evidence from Job Postings and Patent Data, was co-authored by students Qingyu Cao, Qi Guan, Shengzhu Peng, and Po-Yuan Chen, and was presented at the 2025 International AAAI Conference on Web and Social Media (ICWSM), held June 23-26 in Copenhagen, Denmark.
Published on July 7th, 2025
Last updated on July 7th, 2025
The Spanish Society of Medical Radiology (SERAM) has collaborated with six other scientific societies to develop an AI-supported urology clinical decision-making project called Uro-Oncogu(IA)s.
The initiative produced an algorithm that will “reduce time and clinical variability” in the management of urological patients, the society said. SERAM’s collaborators include the Spanish Urology Association (AEU), the Foundation for Research in Urology (FIU), the Spanish Society of Pathological Anatomy (SEAP), the Spanish Society of Hospital Pharmacy (SEFH), the Spanish Society of Nuclear Medicine and Molecular Imaging (SEMNIM), and the Spanish Society of Radiation Oncology (SEOR).
SERAM Secretary General Dr. MaríLuz Parra launched the project in Madrid on 3 July with AEU President Dr. Carmen González.
On behalf of SERAM, the following doctors participated in this initiative:
- Prostate cancer guide: Dr. Joan Carles Vilanova, PhD, of the University of Girona,
- Upper urinary tract guide: Dr. Richard Mast of University Hospital Vall d’Hebron in Barcelona,
- Muscle-invasive bladder cancer guide: Dr. Eloy Vivas of the University of Malaga,
- Non-muscle invasive bladder cancer guide: Dr. Paula Pelechano of the Valencian Institute of Oncology in Valencia,
- Kidney cancer guide: Dr. Nicolau Molina of the University of Barcelona.
-
Funding & Business7 days agoKayak and Expedia race to build AI travel agents that turn social posts into itineraries
-
Jobs & Careers7 days ago
Mumbai-based Perplexity Alternative Has 60k+ Users Without Funding
-
Mergers & Acquisitions7 days ago
Donald Trump suggests US government review subsidies to Elon Musk’s companies
-
Funding & Business6 days ago
Rethinking Venture Capital’s Talent Pipeline
-
Jobs & Careers6 days ago
Why Agentic AI Isn’t Pure Hype (And What Skeptics Aren’t Seeing Yet)
-
Funding & Business4 days ago
Sakana AI’s TreeQuest: Deploy multi-model teams that outperform individual LLMs by 30%
-
Jobs & Careers6 days ago
Astrophel Aerospace Raises ₹6.84 Crore to Build Reusable Launch Vehicle
-
Funding & Business7 days agoFrom chatbots to collaborators: How AI agents are reshaping enterprise work
-
Tools & Platforms6 days ago
Winning with AI – A Playbook for Pest Control Business Leaders to Drive Growth
-
Jobs & Careers4 days ago
Ilya Sutskever Takes Over as CEO of Safe Superintelligence After Daniel Gross’s Exit
