Connect with us

Top Stories

Some people are defending Perplexity after Cloudflare ‘named and shamed’ it

Published

on


When Cloudflare accused AI search engine Perplexity of stealthily scraping websites on Monday, while ignoring a site’s specific methods to block it, this wasn’t a clear-cut case of an AI web crawler gone wild.

Many people came to Perplexity’s defense. They argued that Perplexity accessing sites in defiance of the website owner’s wishes, while controversial, is acceptable. And this is a controversy that will certainly grow as AI agents flood the internet: Should an agent accessing a website on behalf of its user be treated like a bot? Or like a human making the same request?

Cloudflare is known for providing anti-bot crawling and other web security services to millions of websites. Essentially, Cloudflare’s test case involved setting up a new website with a new domain that had never been crawled by any bot, setting up a robots.txt file that specifically blocked Perplexity’s known AI crawling bots, and then asking Perplexity about the website’s content. And Perplexity answered the question.

Cloudflare researchers found the AI search engine used “a generic browser intended to impersonate Google Chrome on macOS” when its web crawler itself was blocked. Cloudflare CEO Matthew Prince posted the research on X, writing, “Some supposedly ‘reputable’ AI companies act more like North Korean hackers. Time to name, shame, and hard block them.”

But many people disagreed with Prince’s assessment that this was actual bad behavior. Those defending Perplexity on sites like X and Hacker News pointed out that what Cloudflare seemed to document was the AI accessing a specific public website when its user asked about that specific website. 

“If I as a human request a website, then I should be shown the content,” one person on Hacker News wrote, adding, “why would the LLM accessing the website on my behalf be in a different legal category as my Firefox web browser?”

A Perplexity spokesperson previously denied to TechCrunch that the bots were the company’s and called Cloudflare’s blog post a sales pitch for Cloudflare. Then on Tuesday, Perplexity published a blog in its defense (and generally attacking Cloudflare), claiming the behavior was from a third-party service it uses occasionally.

Techcrunch event

San Francisco
|
October 27-29, 2025

But the crux of Perplexity’s post made a similar appeal as its online defenders did.

“The difference between automated crawling and user-driven fetching isn’t just technical — it’s about who gets to access information on the open web,” the post said. “This controversy reveals that Cloudflare’s systems are fundamentally inadequate for distinguishing between legitimate AI assistants and actual threats.”

Peplexity’s accusations aren’t exactly fair, either. One argument that Prince and Cloudflare used for calling out Perplexity’s methods was that OpenAI doesn’t behave in the same way.

“OpenAI is an example of a leading AI company that follows these best practices. They respect robots.txt and do not try to evade either a robots.txt directive or a network level block. And ChatGPT Agent is signing http requests using the newly proposed open standard Web Bot Auth,” Prince wrote in his post

Web Bot Auth is a Cloudflare-supported standard being developed by the Internet Engineering Task Force that hopes to create a cryptographic method for identifying AI agent web requests.

The debate comes as bot activity reshapes the internet. As TechCrunch has previously reported, bots seeking to scrape massive amounts of content to train AI models have become a menace, especially to smaller sites. 

For the first time in the internet’s history, bot activity is currently outstripping human activity online, with AI traffic accounting for over 50%, according to Imperva’s Bad Bot report released last month. Most of that activity is coming from LLMs. But the report also found that malicious bots now make up 37% of all internet traffic. That’s activity that includes everything from persistent scraping to unauthorized login attempts.

Until LLMs, the internet generally accepted that websites could and should block most bot activity given how often it was malicious by using CAPTCHAs and other services (such as Cloudflare). Websites also had a clear incentive to work with specific good actors, such as Googlebot, guiding it on what not to index through robots.txt. Google indexed the internet, which sent traffic to sites.

Now, LLMs are eating an increasing amount of that traffic. Gartner predicts that search engine volume will drop by 25% by 2026.  Right now humans tend to click website links from LLMs at the point they are most valuable to the website, which is when they are ready to conduct a transaction.

But if humans adopt agents as the tech industry predicts they will — to arrange our travel, book our dinner reservations, and shop for us — would websites hurt their business interests by blocking them? The debate on X captured the dilemma perfectly:

“I WANT perplexity to visit any public content on my behalf when I give it a request/task!” wrote one person in response to Cloudflare calling Perplexity out.

“What if the site owners don’t want it? they just want you [to] directly visit the home, see their stuff” argued another, pointing out that the site owner who created the content wants the traffic and potential ad revenue, not to let Perplexity take it.

“This is why I can’t see ‘agentic browsing’ really working — much harder problem than people think. Most website owners will just block,” a third predicted.



Source link

Top Stories

SpaceX launches 24 Starlink satellites into polar orbit – Spaceflight Now

Published

on


File: A SpaceX Falcon 9 rocket stands in the vertical position at sunset at Space Launch Complex 4 East (SLC-4E) at Vandenberg Space Force Base. Image: SpaceX

Update Aug. 30, 1:15 a.m. EDT: SpaceX landed the booster on the drone ship.

SpaceX launched 24 Starlink broadband satellites from Vandenberg Space Force Base Friday evening on a Falcon 9 rocket launch.

The mission dubbed Starlink 17-7 was the fourth and final Starlink mission to launch from California in August. SpaceX liftoff of its Falcon 9 rocket from Space Launch Complex 4 East occurred at 9:59 p.m. PDT (12:59 a.m. EDT / 0459 UTC).



SpaceX launched its Starlink satellites using the Falcon 9 first stage booster with the tail number B1082, which flew for a 15th time. It previously launched two national security payloads, a batch of OneWeb broadband Internet satellites and 15 previous batches of Starlink satellites.

Nearly 8.5 minutes after liftoff, B1082 landed on the drone ship, ‘Of Course I Still Love You.’ This was the 147th landing on this vessel and the 496th booster landing to date.

Prior to Friday’s launch, SpaceX launched 27 Starlink missions from Vandenberg in 2025 out of a total of 39 Falcon 9 launches. All told, the company launched 106 orbital missions this year as it aims for 170 such launches by year’s end.

SpaceX continues to bolster its low Earth orbit Starlink constellation as it rolls out the service to more countries and territories around the world. According to numbers tracked by astronomer and expert orbital tracker, Jonathan McDowell, there are more than 8,200 Starlink satellites on orbit.



Source link

Continue Reading

Top Stories

SCO summit: China’s Xi rolls out the red carpet for Putin and Modi as Trump upends global relations

Published

on



Tianjin, China
 — 

Autocrats, populists, friends and foes, a strongman waging a war in Europe and the leader of the world’s biggest democracy will all be hosted by Chinese leader Xi Jinping this weekend at a summit designed to showcase Beijing as a global leader capable of providing a counterweight to Western institutions.

Heads of state and delegations from across Asia and the Middle East will meet from Sunday in the Chinese port city of Tianjin for the two-day summit of the Shanghai Cooperation Organization (SCO), a regional security grouping that has emerged as a cornerstone of Xi and Russian President Vladimir Putin’s drive to rebalance global power in their favor.

Chinese officials have billed the summit as the SCO’s largest yet, with the diplomacy and pageantry setting the stage for Xi to tout his country as a stable and powerful alternative leader at a time when ruling superpower the United States under President Donald Trump is shaking up its alliances and waging a global trade war.

The gathering also comes days ahead of a major military parade in Beijing that will offer a different message – that of China’s rapidly developing military might, and draw autocrats like North Korea’s Kim Jong Un and Myanmar’s Min Aung Hlaing alongside Putin and Russia-friendly European leaders Aleksandar Vucic of Serbia and Slovakia’s Robert Fico.

The summit also gifts Russia’s Putin some international spotlight, just two weeks after his Alaska summit with Trump, and as he continues to ignore international pressure to end his onslaught in Ukraine. Earlier this week Russia’s forces carried out the second-biggest aerial attack since its full-scale invasion of its neighbor.

Ahead of his arrival Putin praised the China-Russia partnership as a “stabilizing force” for the world. In a written interview for Chinese state news agency Xinhua, he said they were “united in our vision of building a just, multipolar world order” – an allusion to the two countries’ efforts to revise what they see as a US-led world order unfairly stacked against them.

SCO members – which include China, Russia, India, Iran, Pakistan, Belarus, Kazakhstan, Kyrgyzstan, Tajikistan, and Uzbekistan – preside over vast swaths of global energy reserves and govern some 40% of the global population.

‘Architect and host’

The guests at the summit have national rivalries between them and vast differences in political systems. And while this has sparked criticism of the group as too disparate to be effective, it also may serve to underscore Xi’s message.

“Beijing wants to signal that China is the indispensable convener in Eurasia, capable of seating rivals at the same table and translating great-power competition into managed interdependence,” said Rabia Akhtar, director of the Centre for Security, Strategy and Policy Research at the University of Lahore in Pakistan.

“The optics are straightforward: China is not just a participant in regional order-making – it is a primary architect and host.”

Modi’s attendance at the gathering also adds heft to Xi’s guest list. The Indian prime minister skipped last year’s summit in Kazakhstan. He now arrives in Tianjin against a backdrop of souring relations with Washington – and as Beijing and New Delhi have moved to ease their own frictions, a nascent realignment that could imperil US efforts to cultivate India as a counterweight against a rising China.

Delegations are also expected from the SCO’s 16 official partner and observer countries, which include Cambodia, Egypt, Saudi Arabia, the United Arab Emirates, Bahrain, Qatar, and Kuwait, as well as NATO member Turkey, among others, Chinese officials said ahead of the gathering.

Beijing additionally invited a handful of Southeast Asian leaders. UN Secretary General Antonio Guterres is also expected to attend.

Across the city of Tianjin, banners in English, Russian and Chinese heralding the gathering lined highways. Officials heavily restricted traffic in the city center as Chinese leaders prepare to welcome their guests with the ceremony and pomp typical of Chinese diplomacy at the highest level.

The location has pointed symbolism for China, as a port that was forced open by colonial powers in the 19th century, with those from Europe and Imperial Japan receiving land concessions – and a key city occupied by Japan during World War II.

Some guests, including Putin, Iranian President Masoud Pezeshkian and Pakistani Prime Minister Shehbaz Sharif, are slated to stay for the military parade in Beijing, where the ruling Communist Party will show off its military might and play up its role fighting Imperial Japan as part of the Allied Forces in World War II as the globe marks 80 years since the end of that war.

Since its formation in 2001 as a group focused on regional security cooperation between China, Russia, Kazakhstan, Kyrgyzstan, Tajikistan, and Uzbekistan, the Shanghai Cooperation Organization has expanded in its size and scope.

SCO members conduct joint counter-terrorism drills, share intelligence on combatting “terrorism, separatism, and extremism,” and work to expand coordination across areas like education, trade and energy. They’re also united in a call for a “just” international order – or one not led by a single superpower and its allies.

Iran’s inclusion in the group in 2023 and Belarus’ a year later have been widely seen as an effort by Beijing and Moscow to the make the body more explicitly anti-West. It also is one facet of a tightening of bonds between Moscow, Beijing and Tehran that’s raised alarm in Washington.

Now, growing frictions and uncertainties between some countries and the US under Trump will loom over the gathering – a reality sure to be referenced prominently if indirectly in remarks by Xi to his guests in the days ahead.

Observers will be watching whether this summit will produce momentum toward further economic integration between member countries, especially when it comes to regional trade or development finance, but expectations for practical developments are low.

“Without deeply addressing what the mission of the Shanghai Cooperation Organization should be and how it can resolve internal sources of conflict both between members and between members with external countries, the SCO is just a showcase,” said Shanghai-based foreign affairs analyst Shen Dingli.

Even as the group regularly calls for “avoiding bloc, ideological or confrontational approaches” to addressing security threats, its summits have yet to produce a joint statement mentioning the war in Ukraine.

SCO – and its lead members China and Russia – also appeared to have little role in de-escalating a four-day conflict between members India and Pakistan earlier this year.

The group did, however, “strongly condemn” the military strikes carried out by the US and Israel on Iran this past June.





Source link

Continue Reading

Top Stories

Taylor Townsend is in the US Open spotlight after Jelena Ostapenko argument

Published

on


NEW YORK (AP) — Taylor Townsend is in the spotlight at the U.S. Open as a result of an interaction she wishes never took place.

Townsend said Jelena Ostapenko told her she had “no class” and “no education” during a face-to-face argument after their second-round match Wednesday. A huge crowd cheered her on in doubles on Thursday, and Townsend made the most of a spot in prime time at Arthur Ashe Stadium on Friday night, upsetting fifth-seeded Mirra Andreeva 7-5, 6-2.

Going into this Grand Slam, Townsend had nowhere near the star power or the name recognition of fellow Americans Coco Gauff and Jessica Pegula, and she is not even seeded in singles play. Yet the 29-year-old who is half of the top-ranked women’s doubles team in the world and was No. 1 as a junior player has become one of the biggest stories of the tournament through no fault of her own.

“It’s bigger than me,” she said on the court after the match. “It’s about the message, it’s about the representation, it’s about being bold and being able to show up as yourself and I did that tonight. You guys saw the real Taylor Townsend tonight.”

Townsend hopes the attention around the confrontation and her calling attention to it can be a positive for the U.S. Open and tennis in general.

“If I’m someone who can draw huge crowds into the stadiums as a name that can bring people to come and buy tickets and support the game, then that’s a crown that I’ll gladly wear,” Townsend said. “Whatever that it is, whatever type of attention that it brought, it’s doing the right things, which is bring people to see the sport and bringing people in to support and that’s what it’s all about.”

Townsend, who is Black, and Ostapenko, who is from Latvia, had an intense back and forth after Townsend won in straight sets. When asked if she thought the comments had racial undertones, Townsend said she didn’t take it that way but acknowledged, “That has been a stigma in our community of being ‘not educated’ and all of the things, when it’s the furthest thing from the truth.”

Gauff and Naomi Osaka were among those who publicly came to Townsend’s defense. Osaka called what Townsend reported Ostapenko saying “one of the worst things you can say to a Black tennis player in a majority white sport.”

Even privately, Townsend said other players came up to her to broach the subject and express their support. Online, she gained thousands of social media followers.

“It’s cool to know that people see you and people are watching and more than anything,” Townsend said. “I was hoping that it was received a certain type of way, and it was, so it was just external validation that I handled things the right way and that’s what I’m the most proud of and the most happy with. I wasn’t looking for that, and in my answers and when I decided and I spoke and I said what I said I wasn’t looking for those things, but it’s nice to know that I made people proud.”

Townsend is in the fourth round at the U.S. Open, more than a decade after the U.S. Tennis Association decided to hold her out of junior competition over concerns about her fitness. The organization in 2012 withheld funding her tournament appearances while she focused on getting in better shape.

In the intervening time, she has become dominant in doubles, winning Wimbledon last year and the Australian Open earlier this year with partner Katerina Siniakova, and the pair is the top seed in Flushing Meadows.

Townsend, who has not gotten past the fourth round in singles at a major, will face two-time Grand Slam singles champion Barbora Krejcikova on Sunday.

If she reaches the quarterfinals this time, she wants the lesson to be that it is OK to stand up for yourself.

“Sometimes I feel like in society, especially people of color, we are expected to be silenced, or sometimes there are times where we have to decide and be very strategic as to when we speak up, and in these type of moments, it’s important for me to speak up, not only for myself but for my culture,” Townsend said. “No matter what, no matter what attention comes or whatever, I think it’s about being unapologetically yourself, be happy in who you are and never allow anyone to take you out of your character and who you are as a person.”

___

AP tennis: https://apnews.com/hub/tennis





Source link

Continue Reading

Trending