There is an important and ubiquitous process in scientific research that scientists never seem to talk about. There isn't even a word for it, as far as I now, so I'll introduce my own: I'll call it knowledge distillation.
What is Knowledge distillation? | IBM
Knowledge distillation is a machine learning technique used to transfer the learning of a large pre-trained “teacher model” to a smaller “student model.”
Konrad Hinsen's blog
Knowledge refinement is the ever ongoing process in science (and beyond it) that shepherds knowledge from lab notebooks into journal articles and then on to review articles, monographs, reference handbooks, university textbooks, and finally professional domain expertise and school education for a wider public. It has been going on for a few centuries, but we hardly talk about it. In fact, I made up the term because I couldn't find an established one. Computational knowledge has not yet found its place in the knowledge refinement process. Why not? And what can we do to make it happen?
Everything You Need to Know about Knowledge Distillation
A Blog post by Ksenia Se on Hugging Face
Science Must Decentralize
Knowledge production doesn’t happen in a vacuum. Every great scientific breakthrough is built on prior work, and an ongoing exchange with peers in the field. That’s why we need to address the threat
The Bazaar of Scientific Knowledge | shishyko!
What if we didn't collapse all the knowledge from the scientific process into one paper?
On-Policy Distillation
On-policy, dense supervision is a useful tool for distillation
See what you think
Allegra A. Beal Cohen's blog about knowledge curation, new interfaces, and large-scale qualitative data.
Distilling the Knowledge in a Neural Network
Our teams advance the state of the art through research, systems engineering, and collaboration across Google.
Managing scientific knowledge for policy on the ATmosphere - Mathew's newsletter
Two types of Atmosphere toolkit are needed if ATScience is to help scientists do science and better communicate it to other audiences: one serving the researchers and their teams, the other focused on aggregation and synthesis.
Universal Scientific Protocols, Inc.
Universal Scientific Protocols, Inc. — research publishing, reconsidered. A knowledge management platform for machine learning researchers.
DIKW pyramid
The DIKW pyramid is a model describing relationships between data, information, knowledge and wisdom sometimes also stylized as a chain, refer to models of possible structural and functional relationships between a set of components—often four: data, information, knowledge, and wisdom. The concept has roots predating the 1980s. In the latter years of that decade, interest in the models grew after explicit presentations and discussions, including from Milan Zeleny, Russell Ackoff, and Robert W. Lucky. Subsequent important discussions extended along theoretical and practical lines into the coming decades.
The Engine of Scientific Discovery: How New Methods and Tools Spark Major Breakthroughs
Abstract. How do we spark new scientific discoveries? Why do some breakthroughs seem even accidental? And most importantly, how can we accelerate them and
The Bazaar of Scientific Knowledge | shishyko!
Is the current form of the scientific paper still optimal in 2025? How do we preserve, and efficiently leverage, the uncut gems of the scientific process?
Coordination Tech in Science: Letters, Journals, and Whatever Comes Next | shishyko!
To modernize our scientific infrastructure, we need new contextualization and coordination technologies that decouple trust from legacy branding — shifting from gatekeeping on write to algorithmic contextualization on read.