Operadic consistency: a label-free signal for compositional...
Detecting LLM reasoning failures at inference time without ground-truth labels has motivated a wide range of confidence baselines, including self-consistency, semantic entropy, and P(True), built...
EsDictionary - The EsDeeKid Translator
Cross-Lingual Exploration for Parametric Knowledge
Parametric knowledge in Large Language Models is not equally accessible across languages. As a result, standard inference techniques often struggle to surface localized facts, leading to failures in...
The Lambek Calculus
There is a noticeable revival of categorial grammar these days, as a vehicle for linguistic description. The systems used differ somewhat from the original calculus of Ajdukiewicz and Bar-Hillel, however. In particular, there is a component of rules for ‘type change’ of expressions, making for greater flexibility and elegance. One fundamental system of this kind is the so-called ‘Lambek Calculus’, whose type-change rules show a close analogy with the inference rules of constructive propositional logic. In this paper, we present one calculus of this kind, and survey its theoretical properties as a device in linguistic semantics. Our two main new contributions are a new and complete semantics for this calculus, as well as a modest study of its language-accepting capacity. In this way, we hope to provide a better understanding of the background theory of flexible categorial grammar, in tandem with its descriptive uses.
Probabilistic compositional semantics, purely
We provide a general framework for the integration of formal semantics with probabilistic reasoning. This framework is conservative, in the sense that it relies only on typed λ-calculus and is thus co
Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.
Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.
Inference characteristics of Llama · Cursor
A primer on inference math and an examination of the surprising costs of Llama.
Producing The Perfect Token
The unspoken inference quality gap and how numerics determine if the inference you're paying for is worth it.
Finite-Choice Logic Programming | Proceedings of the ACM on Programming Languages
Logic programming, as exemplified by datalog, defines the meaning of a program as its unique smallest model: the deductive closure of its inference rules. However, many problems call for an enumeration of models that vary along some set of choices while ...
Crafting a good (reasoning) model
A recent talk I gave on model training, reasoning, and the next frontier.
The Session You Cannot Take With You | EARENDIL
Inference APIs are filling sessions with encrypted reasoning, hidden search results, opaque compaction, and encrypted subagent messages. A growing form of lock-in.
The Principal Type-Scheme of an Object in Combinatory Logic
R. Hindley, The Principal Type-Scheme of an Object in Combinatory Logic, Transactions of the American Mathematical Society, Vol. 146 (Dec., 1969), pp. 29-60
On the Paradox of Learning to Reason from Data
Logical reasoning is needed in a wide range of NLP tasks. Can a BERT model be trained end-to-end to solve logical reasoning problems presented in natural language? We attempt to answer this...