Controlled natural languages (CNLs) are subsets of natural languages that are constructed by restricting the grammar and vocabulary in order to reduce or eliminate ambiguity and complexity. Traditionally, controlled languages fall into two major types: those that improve readability for human readers (e.g. non-native speakers), and those that enable reliable automatic semantic analysis of the language.[1][2]
What Kind of Programming is Natural Language Programming?
In previous posts I’ve written about Natural Language Programming, Dijkstra’s Ghost – the End of The Symbolic Supremacy and Ephemeral Editable Specifications (aka Extract, Edit, A…
Controls as language
A couple of weeks ago, my friend Esteban said some stuff on Twitter that got me thinking (again) about the value of looking at a game’s control scheme as a sort of language. The whole series of tweets is worth a read, but the concluding tweet in p...
Simply Typed Reverse-Mode Automatic Differentiation with Variants: Denotational Correctness via Idempotent Completion
Reverse-mode automatic differentiation can be derived denotationally as a structure-preserving interpretation of program syntax. In the usual simply typed model, each source type has one cotangent type. Variants break this representation because the valid cotangent space depends on the branch selected at run time; established correctness results therefore use primal-indexed families of cotangent spaces, whose direct internal language is dependent.
ICML Poster Learning syntax without semantics: Disentangled tiny language models
Language models acquire syntax and world knowledge together, entangling the two in ways that limit efficiency and controllability. We show that syntax can be learned while suppressing semantic plausibility and world‑knowledge cues, yielding more efficient and controllable models. We train tiny LMs on grammatical nonsense — syntactically well-formed text with semantic content ablated via constrained relexicalization (SAMBAL). Models trained on this data perform comparably to standard pretraining on syntactic benchmarks (BLiMP, SyntaxGym) while scoring at chance on world knowledge probes (EWoK). On targeted grammar-plausibility conflict probes, content-neutral models prefer grammaticality where standard models prefer plausibility, and their representations show more syntactic vs lexical alignment. On efficiency, disentanglement yields substantial sample and parameter gains: in low‑resource regimes, a 5M‑parameter model matches a 30M‑parameter baseline at the same data budget. On controllability, content-neutral models adapt rapidly to a new domain with minimal exposure, suggesting the feasibility of modular post‑hoc knowledge specialization.
Large language models are cultural technologies. What might that mean?
Four different perspectives
An Intuitive Introduction to Flow-Based Language Generation — Floor Eijkelboom
Why autoregressive models have a structural ceiling, why discrete-diffusion LLMs are still autoregression in disguise, and what flow-based language models actually give us.
I trained a cellular automaton to write English, using no neural net at all
Each word or letter is written into a grid of integer cells. Learned rules rewrite the cells, and the next word is read from the grid. You can watch…
There are no lossless transformations of natural-language text
In my work at Clay I recently wrote an internal policy on acceptable use of AI writing by engineers, and I’m sharing it here. It’s my hope that one day better AI tools might be able to help us think, but until then I fear that using AI to write does the exact opposite.
agentlanguages.dev — Programming languages designed for AI agents to write
A community-edited catalogue of programming languages designed for AI agents to author code, organised around three philosophical camps: syntactic, verification, and orchestration.
Recursive Language Models: the paradigm of 2026
How we plan to manage extremely long contexts
.png?v=c8c07d4bf43b)
Slim Lim: "Concrete syntax matters, actually"