A latent space, also known as a latent feature space or embedding space, is an embedding of a set of items within a manifold in which items resembling each other are positioned closer to one another. Position within the latent space can be viewed as being defined by a set of latent variables that emerge from the resemblances between the objects.
Latent Space as a New Medium
Lately I’ve been asking myself: what might artificial intelligence be good for besides answering questions and writing code?
Manifold hypothesis
The manifold hypothesis posits that many high-dimensional data sets that occur in the real world actually lie along low-dimensional latent manifolds inside that high-dimensional space.[1][2][3][4] As a consequence of the manifold hypothesis, many data sets that appear to initially require many variables to describe, can actually be described by a comparatively small number of variables, linked to the local coordinate system of the underlying manifold. It is suggested that this principle underpins the effectiveness of machine learning algorithms in describing high-dimensional data sets by considering a few common features.
Approaching an unknown communication system by latent space exploration and causal inference | Royal Society Open Science | The Royal Society
Abstract. We propose a methodology for discovering meaningful properties in data without ground truth by combining manipulation of the latent variables of
Word embedding
In natural language processing, a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning. Word embeddings can be obtained using language modeling and feature learning techniques, where words or phrases from the vocabulary are mapped to vectors of real numbers.
Latent Spacecraft: Brains, GANs, Finnegans.
Latent Spacecraft combines computational linguistics, neuroscience, and literary analysis to investigate latent space, i.e. the hidden internal structure that enables both humans and machines to produce language. Peeking into AI’s hidden interiority, we parallel speech generation in humans and speech-trained generative adversarial networks (GANs), as well as in the language of Joyce’s Finnegans Wake and the GAN model trained on the novel, FinneGAN.
Factored Latent Action World Models
Learning latent actions from action-free video has emerged as a powerful paradigm for scaling up controllable world model learning. Latent actions provide a natural interface for users to...
No Space Like J-Space
There is a new very cool Anthropic paper: Verbalizable Representations Form a Global Workspace in Language Models. You can read the blog post verison here.
On Interpolatable Archives
AI Latent Spaces as Shapeshifting Skeleton Libraries and Explosion Drawings bearing cognitive hazards and new opportunities to play.
Top2Vec: Distributed Representations of Topics
Topic modeling is used for discovering latent semantic structure, usually referred to as topics, in a large collection of documents. The most widely used methods are Latent Dirichlet Allocation and Probabilistic Latent…
Meandering on Manifolds: The Neural Geometry of Stories Over Time
To fully understand LLM representations, we must understand how they change dynamically, over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent human emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)?
Meandering on Manifolds: The Neural Geometry of Stories Over Time
To fully understand LLM representations, we must understand how they change dynamically, over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent human emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)?
The Beginner’s Guide to Text Embeddings & Techniques | deepset Blog
Text embeddings represent human language to computers, enabling tasks like semantic search. Here, we introduce sparse and dense vectors in a non-technical way.
New paper finally out in @NatureComms with E. Leib, D. O’Shaughnessy, C. Gallardo, @sferrigno.bsky.social, and @spiantado.bsky.social. 📝Children across cultures discover the latent algorithms that structure what they see, even without instruction, feedback, or formal schooling.🧵 tinyurl.com/4a238m2d
