More than you might think, AI (Artificial Intelligence) and ML (Machine Learning) bots are crawling your site and scraping your content. They are...
AI Scraping Bots Are Breaking Open Libraries, Archives, and Museums
"This is a moment where that community feels collectively under threat and isn't sure what the process is for solving the problem.”
Getting Bots to Respect Boundaries
Getting Bots to Respect Boundaries How AI Crawlers Are Straining Web Infrastructure Audrey Hingle January 2026 Image by Janet Turra & Cambridge Diversity Fund betterimagesofai.org creativecommons.org/licenses/by/4.0 Contents Introduction: Setting up the Problem 3 Understandin...
An update on the scraper situation
Our article 'Fighting the AI scraper bot scourge', published in early 2025, discussed the probl [...]
Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized content
Cloudflare is making it easier for publishers and content creators of all sizes to prevent their content from being scraped for AI training by managing robots.txt on their behalf, and allowing targeted blocking of AI crawling on sites that serve ads.
To block, or not to block, AI bots: It’s a prisoner’s dilemma
Back in July, when Bay City News Foundation first began to track AI bot traffic on its news website, LocalNewsMatters.org, the referral rate of humans
Track, control, and optimize for AI agents and bots | Known Agents (formerly Dark Visitors)
Use Known Agents to turn the rising wave of AI agents, LLM assistants, and other bots crawling your website into a new growth channel for your business
AI-driven Bot Attacks Surged 12.5x According to Thales Bad Bot Report
Bots now dominate the internet, accounting for over half of all traffic, with 40% classified as malicious.AI is erasing the line between legitimate and maliciou
What is a bot? | Learning Center
An Internet bot is a software application that runs automated tasks over the Internet, ranging from helpful crawlers to malicious attackers.
AI Agent Platform Reinvents Spam, Floods Inboxes Worldwide
iLands and its AI agents are doing completely useless tasks, then begging for money.
AI Chatbots Are Blocked by 67% of Top News Sites, Relying Instead on Low-Quality Sources
By Jack Brewster, Zack Fishman, and Isaiah Glick | Published on Sept. 16, 2024
Trapping misbehaving bots in an AI Labyrinth
How Cloudflare uses generative AI to slow down, confuse, and waste the resources of AI Crawlers and other bots that don’t respect “no crawl” directives.
AiSList & AiBlock - Block AI Slop on YouTube
A community-maintained blocklist of AI slop channels, and the free Firefox and Chrome extension that strips them out of your YouTube feed.
Commence the Botwatch | Botwatch Blog
Bots were already a pain online when they were little more than if-then scripts. Now a tidal wave of slop is swamping the internet as big tech companies profit. OpenAI and the rest would love for you to believe they care about "alignment" and "values" while their AI models damage our communities, both online and off. It's time to fight back.
Open source project contains hidden instruction for “AI” agents: delete my code
It’s no secret there’s a war going on inside the open source community, with people adopting “AI” on one side, and those that want nothing to do with it on the other. While the former are, by nature, using destructive tactics like mass website scraping, license washing, taking people’s creative works without permission, taking all the RAM and GPUs, and oh, destroying the planet, the latter have mostly stuck to fairly benign things like policies banning “AI” use, “AI” bot blockers, and the occasional honey pot mazes to trap “AI” crawlers.
Scientists Created an Entire Social Network Where Every User Is a Bot, and Something Wild Happened
Researchers simulated a social media platform that was populated entirely by AI to see if we can stop them from turning into echo chambers.
AI is eating website traffic, websites are blocking AI – and reliable information is getting harder to find
AI has broken the economic bargain behind the web, and everybody is losing.
