Understanding LSTM Networks -- colah's blog

Long short-term memory

Long short-term memory (LSTM) is a type of recurrent neural network (RNN) aimed at mitigating the vanishing gradient problem commonly encountered by traditional RNNs. Its relative insensitivity to gap length is its advantage over other RNNs, hidden Markov models, and other sequence learning methods. It aims to provide a short-term memory for RNN that can last thousands of timesteps. The name is made in analogy with long-term memory and short-term memory and their relationship, studied by cognitive psychologists since the early 20th century.

Human Routers of Machine Words

On people who use AI to think.

Large language model

A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts, and are a foundational technology behind modern chatbots.[1] Biased or inaccurate training data can make an LLM's output less reliable.[2]

Meandering on Manifolds: The Neural Geometry of Stories Over Time

To fully understand LLM representations, we must understand how they change dynamically, over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent human emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)?

Meandering on Manifolds: The Neural Geometry of Stories Over Time

To fully understand LLM representations, we must understand how they change dynamically, over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent human emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)?

We Don't Understand Neural Networks At The Algorithmic Level

The largest ongoing debate about AI is “Are Large Language Models (LLMs) intelligent?” That makes sense, at least: the evidence is ambiguous and the stakes a...

Illusions of Understanding from Outsourcing Thinking to LLMs

Some illusions of understanding are an inevitable part of the research process, while others can be avoided or overcome by careful critical thinking and observation. We are facing an increased risk of avoidable illusions as more research activities are delegated to large language models (LMM). LLMs can be useful but they cannot think, and their use can undermine our thinking and understanding. Thinking for ourselves is hard and error prone but worthwhile - and there are no shortcuts to understanding.

Take caution in using LLMs as human surrogates | PNAS

Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, survey...

Basic Memory

AI conversations that actually remember. Never re-explain your project to your AI again. Join our Discord: https://discord.gg/tyvKNccgqN

Scaffolding, Hard and Soft: Infrastructures as Critical and Generative Structures

Words in Space is the work of Shannon Mattern.

Recurrent neural network

In artificial neural networks, recurrent neural networks (RNNs) are designed for processing sequential data, such as text, speech, and time series,[1] where the order of elements is important. Unlike feedforward neural networks, which process inputs independently, RNNs utilize recurrent connections, where the output of a neuron at one time step is fed back as input to the network at the next time step. This enables RNNs to capture temporal dependencies and patterns within sequences.

How AI Is Learning to Think in Secret — LessWrong

On Thinkish, Neuralese, and the End of Readable Reasoning • ---------------------------------------- …

Thinking modes

Floating in linguistic space.