Making web crawlers eat shit since 2023
Aggressive AI scrapers are making it kinda suck to run wikis
Bots are currently scraping the internet for LLM training data at unprecedented rates[1][2][3], driving up costs and destabilizing public-facing websites. I want to talk about how this has been particularly difficult for wikis, and has gotten much worse in the last few months.
Sustainable Web Manifesto
“If the Internet was a country, it would be the 13th largest polluter”
zio.sh/didweb.watch
🦋🤖 Bluesky bot to track did:web usage — https://bsky.app/profile/didweb.watch
Saying “No” In an Age of Abundance
Writing about the big beautiful mess that is making things for the world wide web.
The reckless, infinite scope of web browsers
This blog post was published on March 18, 2020. It might be out of date, have broken images or links, or simply contain bad ideas and opinions that didn't stand the test of time. Please read generously!
An update on the scraper situation
Our article 'Fighting the AI scraper bot scourge', published in early 2025, discussed the probl [...]
The Web has lost its Soul - LOVEWEB
Anti-Crawler Defenses | The Paper Pilot
The Paper Pilot's digital garden of thoughts on philosophy, politics, and sociology
Self-Hosting on the Dark Web | David Álvarez Rosa
This site is now reachable over Tor as a hidden service, at a .onion address that resolves only inside the Tor network. Tor relays and encrypts your traffic as …
The Feed is the Product: Why We Stay in Walled Gardens, and Why We Might Finally Leave for the Open Social Web - Zsolt's pckt blog
We have known this for years. We have known that the major platforms are bad for our attention, our sleep, our politics, our children, and our sense of reality....
Funding The Web: A Wise Choice
The Web is slow-motion collapsing for multiple reasons, but at the heart of it all is a little known system: the Search/Browser Levy. Let's fix it!
being utopian about the web is very 90s but this is like everything people dreamed the web would be. I need to read more about atproto asap overreacted.io/open-social/
Cosmik Network
Semble alpha is now open! ✨ Check it out at semble.so More details in our leaflet pub! (link in next post > )
