DisCoCat (Categorical Compositional Distributional) is a mathematical framework for natural language processing which uses category theory to unify distributional semantics with the principle of compositionality. The grammatical derivations in a categorial grammar (usually a pregroup grammar) are interpreted as linear maps acting on the tensor product of word vectors to produce the meaning of a sentence or a piece of text. String diagrams are used to visualise information flow and reason about natural language semantics.
A multiclass Q-NLP sentiment analysis experiment using DisCoCat
Sentiment analysis is a branch of Natural Language Processing (NLP) which goal is to assign sentiments or emotions to particular sentences or words. Performing this task is particularly useful for...
Categorial grammar
Categorial grammar is a family of formalisms in natural language syntax that share the central assumption that syntactic constituents combine as functions and arguments. Categorial grammar posits a close relationship between the syntax and semantic composition, since it typically treats syntactic categories as corresponding to semantic types. Categorial grammars were developed in the 1930s by Kazimierz Ajdukiewicz and in the 1950s by Yehoshua Bar-Hillel and Joachim Lambek. It saw a surge of interest in the 1970s following the work of Richard Montague, whose Montague grammar assumed a similar view of syntax. It continues to be a major paradigm, particularly within formal semantics.
A categorical manifesto
This paper tries to explain why and how category theory is useful in computing science, by giving guidelines for applying seven basic categorical concepts: category, functor, natural transformation, limit, adjoint, colimit and comma category. Some examples, intuition, and references are given for each concept, but completeness is not attempted. Some additional categorical concepts and some suggestions for further research are also mentioned. The paper concludes with some philosophical discussion.
Category: The Essence of Composition
I was overwhelmed by the positive response to my previous post, the Preface to Category Theory for Programmers. At the same time, it scared the heck out of me because I realized what high expectati…
The Lambek Calculus
There is a noticeable revival of categorial grammar these days, as a vehicle for linguistic description. The systems used differ somewhat from the original calculus of Ajdukiewicz and Bar-Hillel, however. In particular, there is a component of rules for ‘type change’ of expressions, making for greater flexibility and elegance. One fundamental system of this kind is the so-called ‘Lambek Calculus’, whose type-change rules show a close analogy with the inference rules of constructive propositional logic. In this paper, we present one calculus of this kind, and survey its theoretical properties as a device in linguistic semantics. Our two main new contributions are a new and complete semantics for this calculus, as well as a modest study of its language-accepting capacity. In this way, we hope to provide a better understanding of the background theory of flexible categorial grammar, in tandem with its descriptive uses.
Category Theory for Programmers: The Preface
Table of Contents Part One Category: The Essence of Composition Types and Functions Categories Great and Small Kleisli Categories Products and Coproducts Simple Algebraic Data Types Functors Functo…
What Kind of Programming is Natural Language Programming?
In previous posts I’ve written about Natural Language Programming, Dijkstra’s Ghost – the End of The Symbolic Supremacy and Ephemeral Editable Specifications (aka Extract, Edit, A…
Conceptual mathematics: a first introduction to categories
Conceptual mathematics by F. W. Lawvere, 2009, Cambridge University Press edition, in English - 2nd ed.
F. William Lawvere, Stephen H. Schanuel Conceptual Mathematics A First Introduction To Categories ( 2009, Cambridge University Press) ( 1)
Category Theory first introduction
Seven Sketches in Compositionality: An Invitation to Applied Category Theory
This book is an invitation to discover advanced topics in category theory through concrete, real-world examples. It aims to give a tour: a gentle, quick introduction to guide later exploration. The tour takes place over seven sketches, each pairing an evocative application, such as databases, electric circuits, or dynamical systems, with the exploration of a categorical structure, such as adjoint functors, enriched categories, or toposes. No prior knowledge of category theory is assumed. A feedback form for typos, comments, questions, and suggestions is available here: https://docs.google.com/document/d/160G9OFcP5DWT8Stn7TxdVx83DJnnf7d5GML0_FOD5Wg/edit
Categories for Machine Learning
This seminar series seeks to promote the learning and use of Category Theory by Machine Learning Researchers
Categorical quantum mechanics
Categorical quantum mechanics is the study of quantum foundations and quantum information using paradigms from mathematics and computer science, notably monoidal category theory. The primitive objects of study are physical processes, and the different ways these can be composed. It was pioneered in 2004 by Samson Abramsky and Bob Coecke. Categorical quantum mechanics is entry 18M40 in MSC2020.
Categories for the Working Mathematician
Categories for the Working Mathematician provides an array of general ideas useful in a wide variety of fields. Starting from the foundations, this book illuminates the concepts of category, functor, natural transformation, and duality. The book then turns to adjoint functors, which provide a description of universal constructions, an analysis of the representations of functors by sets of morphisms, and a means of manipulating direct and inverse limits. These categorical concepts are extensively illustrated in the remaining chapters, which include many applications of the basic existence theorem for adjoint functors. The categories of algebraic systems are constructed from certain adjoint-like data and characterized by Beck's theorem. After considering a variety of applications, the book continues with the construction and exploitation of Kan extensions. This second edition includes a number of revisions and additions, including two new chapters on topics of active interest. One is onsymmetric monoidal categories and braided monoidal categories and the coherence theorems for them. The second describes 2-categories and the higher dimensional categories which have recently come into prominence. The bibliography has also been expanded to cover some of the many other recent advances concerning categories.
Word embedding
In natural language processing, a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning. Word embeddings can be obtained using language modeling and feature learning techniques, where words or phrases from the vocabulary are mapped to vectors of real numbers.
Language model harnesses are compositional generalizers
Harnesses can lead to compositional generalization: we observe a property in training RLMs, in which similarly structured tasks are viewed as isomorphic and all individual LM calls in the harness become in-distribution.
