A Blog post by Ksenia Se on Hugging Face
What is Knowledge distillation? | IBM
Knowledge distillation is a machine learning technique used to transfer the learning of a large pre-trained “teacher model” to a smaller “student model.”
Ellora: Enhancing LLMs with LoRA - Standardized Recipes for Capability Enhancement
A Blog post by Asankhaya Sharma on Hugging Face
The Best Tacit Knowledge Videos on Every Subject — LessWrong
TL;DR Tacit knowledge is extremely valuable. Unfortunately, developing tacit knowledge is usually bottlenecked by apprentice-master relationships. Ta…
Simply Jekyll
A non-linear thread of important ideas weaved together to explore and understand the fabric of knowledge.
See what you think
Allegra A. Beal Cohen's blog about knowledge curation, new interfaces, and large-scale qualitative data.
Simplifying Zettelkasten by working out loud
What did I learn about learning as I used Zettelkasten idea and knowledge management to write five newsletters about disinformation in…
Why We Built VIBE Bench: Rethinking Evaluation for Real Workloads
A Blog post by MiniMax on Hugging Face
The distillation panic
‘Distillation attacks’ is a horrible term for what is happening right now.
I Built a Knowledge Base That Writes Itself. Here Is What Andrej Karpathy Got Right.
Andrej Karpathy posted about using LLMs to build personal knowledge bases. I took his workflow, wired it into my Obsidian vault with Claude Code, and within an hour had 21 cross-linked wiki articles compiled from YouTube transcripts. Here is how it works and why it matters.
Ham Huang on Twitter / X
🚨New preprint!🚨Excited to share my 1st PhD project with @BonanZhao @cocosci_lab @natvelaliTeaching is a powerful way to pass on "tools for thought" to solve new problems. How well do existing teaching models of teaching capture this process?Link: https://t.co/nX4SbsCPXU— Ham Huang (@Huang_Ham) August 28, 2024
On-Policy Distillation
On-policy, dense supervision is a useful tool for distillation
Wait… whats a Community of Practice?
“Learning is not merely the acquisition of knowledge by individuals. It is a process of social participation.” A slightly bastardised…
Seams - Wisdom is made Together