bgbg
The ability for language models to write coherent text can be explained as each token develops some affinity for other tokens. During next token prediction each of the previous tokens "vote" for the tokens they have high affinity for, which results in coherent continuation. (twitter.com) — Semble