The Web is, without a doubt, the most powerful research tool currently available to man. No longer must researchers comb through endless indices and catalogues to find what they are looking for. Instead, hyperlinks allow users to associatively explore rich interconnected structures of information with ease. In just a few clicks, it is possible to navigate through a vast global space of information comprising documents on virtually any subject.
Funding The Web: A Wise Choice
The Web is slow-motion collapsing for multiple reasons, but at the heart of it all is a little known system: the Search/Browser Levy. Let's fix it!
Break up the Browsers. A Proposal to Save the Open Web. - Movement For An Open Web
The Open Web is one of the world’s modern wonders. Based on core, universally recognised technical standards and owned and controlled by no one body, it is a unique structureRead More
The reckless, infinite scope of web browsers
This blog post was published on March 18, 2020. It might be out of date, have broken images or links, or simply contain bad ideas and opinions that didn't stand the test of time. Please read generously!
Opinion | How to Save the Web (Published 2018)
Governments, businesses and users around the world must commit to a set of fundamental principles for the web.
What the ephemerality of the Web means for your hyperlinks
Hyperlinks are a powerful tool for journalists and their readers. Diving deep into the context of an article is just a click away. But hyperlinks are a double-edged sword; for all of the internet’s boundlessness, what’s found on the Web can also be modified, moved, or entirely vanished. The fragility of the Web poses an […]
Solutions for Link Rot on the Modern Web
Web links are fundamental to the web, enabling navigation between pages and citations in research articles. However, web links suffer from "link rot", a phenomenon in which links are likely to become inaccessible over time. This can occur if a link’s site disappears, making it impossible to resolve the hostname or establish a server connection, or the linked page has been deleted, resulting in an HTTP 404 "Not Found" error. Today, a common solution to tackle link rot is to rely on web archives, which capture snapshots of web pages for future reference. However, the modern web has evolved significantly since web archives were first introduced, leading to several limitations in their effectiveness. First, the scale of the web has grown tremendously, making it infeasible to crawl every page whenever it changes. As a result, many broken links either have no archived copies, or the archived copies have stale content. Second, modern web pages rely heavily on increasingly complex and diverse JavaScript. This shift has made it more challenging for preserving fidelity in archived copies, significantly increasing both the computational cost of operating browser-based crawlers and engineering effort required to maintain accurate replay systems. This thesis presents a set of solutions to cope with link rot on the modern web. My work aims to mitigate the various limitations of web archives. First, for broken links without any archived copy, or if the archived copy includes stale content or unavailable functionalities, I built Fable. Fable revives the dead link with the new URL to the same page whenever available. Compared to prior approaches, Fable revives 4.6K broken links—a 50% increase—with much higher accuracy. Second, for pages that require dynamic crawling, I show how web archives can achieve a better tradeoff between efficiency and fidelity. By carefully choosing 8.9% of pages to crawl dynamically and strategically reusing resources from those crawls, an archive can serve 99% of the remaining statically crawled pages without any fidelity loss. Third, for fidelity violations in archived copies that are caused by incorrect edits to crawled scripts, I built FidEx. FidEx reliably detects when an archived page differs from its original version and pinpoints the root cause. After fixing the most common errors pinpointed by FidEx, I reduced the fraction of pages for which FidEx reports a violation of fidelity from 15% to 9%.
Solutions for Link Rot on the Modern Web
Web links are fundamental to the web, enabling navigation between pages and citations in research articles. However, web links suffer from "link rot", a phenomenon in which links are likely to become inaccessible over time. This can occur if a link’s site disappears, making it impossible to resolve the hostname or establish a server connection, or the linked page has been deleted, resulting in an HTTP 404 "Not Found" error. Today, a common solution to tackle link rot is to rely on web archives, which capture snapshots of web pages for future reference. However, the modern web has evolved significantly since web archives were first introduced, leading to several limitations in their effectiveness. First, the scale of the web has grown tremendously, making it infeasible to crawl every page whenever it changes. As a result, many broken links either have no archived copies, or the archived copies have stale content. Second, modern web pages rely heavily on increasingly complex and diverse JavaScript. This shift has made it more challenging for preserving fidelity in archived copies, significantly increasing both the computational cost of operating browser-based crawlers and engineering effort required to maintain accurate replay systems. This thesis presents a set of solutions to cope with link rot on the modern web. My work aims to mitigate the various limitations of web archives. First, for broken links without any archived copy, or if the archived copy includes stale content or unavailable functionalities, I built Fable. Fable revives the dead link with the new URL to the same page whenever available. Compared to prior approaches, Fable revives 4.6K broken links—a 50% increase—with much higher accuracy. Second, for pages that require dynamic crawling, I show how web archives can achieve a better tradeoff between efficiency and fidelity. By carefully choosing 8.9% of pages to crawl dynamically and strategically reusing resources from those crawls, an archive can serve 99% of the remaining statically crawled pages without any fidelity loss. Third, for fidelity violations in archived copies that are caused by incorrect edits to crawled scripts, I built FidEx. FidEx reliably detects when an archived page differs from its original version and pinpoints the root cause. After fixing the most common errors pinpointed by FidEx, I reduced the fraction of pages for which FidEx reports a violation of fidelity from 15% to 9%.
Web bloat isn’t a knowledge problem | Christian Heilmann
There is a consensus among all browser makers and lovers of the web: the current state of the web is a mess. The average web page is 2.4 megabyte big and has over 200 requests.
The Subversive Hyperlink
Writing about the big beautiful mess that is making things for the world wide web.
Introducing Ringspace: A Proposal for the Human Web
For months, I've been working on a project to demonstrate how we can preserve humanity on the web. It's finally ready for testing.
Funding The Web: From Cartel to Covenant
The Web is slow-motion collapsing for multiple reasons, but at the heart of it all is a little known system: the Search/Browser Levy. Let's fix it!
LN 040: The venerable hyperlink
Decades after hyperlinks transformed humanity's information and text into a densely-linked graph, mainstream OSs have yet to afford users their benefits in personal computing.
Matthew Rothberg a website that deletes itself after it detects it's been indexed by a search engine github.com/mroth/unindexed
GitHub - mroth/unindexed: :mag_right::grey_question: website that irrevocably deletes itself once indexed
github.com