Publishers, Cloudflare Accuse Google of ‘Bundled Crawling’ in AI Race

[tradelikepro]

Media companies and Cloudflare are criticizing Google for what they call “bundled crawling” — using the same Googlebot to scrape websites for both search and AI training, leaving publishers little choice but to comply.

Neil Vogel, CEO of People, Inc., said blocking Google’s crawler “breaks ad systems,” effectively locking publishers into the system.

Cloudflare, which recently launched a tool to let websites charge AI bots for access, shared data with UK regulators showing why Google’s setup is “nearly impossible to replicate,” giving it an unfair advantage over other AI firms like OpenAI and Anthropic.

[Rimon Khan]

Concerns over Google’s AI data scraping show how competition is redefining digital ethics.

[Abdul Khan]

The pushback from publishers signals a growing demand for transparency in AI data collection.