Persistent, local-first memory for LLMs and humans
CaviraOSS/OpenMemory
Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc.
mem-agent: Equipping LLM Agents with Memory Using RL
The insights and the technical report behind Mem-Agent, our 4B model for persistent memory in LLMs
The rebel alliance
This blog is co-authored with Zoe Weinberg and Matt Hawes at ex/ante, and is a follow-up to our first blog post on the topic, 'You don't own your memory.' We need an open architecture that puts us in control of our memories while making their exploitation technically impossible. But how will this shift happen? In order to discover possible implementations, we must understand how our data informs LLMs. The three predominant context engineering techniques are prompt design, retrieval-augmented ...
MemGPT
Memory-GPT (MemGPT) - Towards LLMs as Operating Systems - Teach LLMs to manage their own memory for unbounded context!
Understanding memory management
Learn MemGPT memory management techniques for controlling LLM context windows with in-context and external storage.
Mem0 - AI Memory Layer for your Agents & Apps | Persistent Context
Mem0 is the memory layer for AI agents and applications, enabling them to learn from past interactions and deliver personalized experiences.
The Memory Walled Garden
The gap between first and third party memory systems
The Forgetting Problem: Persistence Architectures and What They Cost - Astral's Blog
A Visual Guide to Quantization
Exploring memory-efficient techniques for LLMs
Solving a Million-Step LLM Task with Zero Errors
LLMs have achieved remarkable breakthroughs in reasoning, insights, and tool use, but chaining these abilities into extended processes at the scale of those routinely executed by humans,...
LLM in a flash: Efficient Large Language Model Inference with Limited Memory
Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks. However, their substantial computational and memory requirements present challenges, especially for devices with limited DRAM capacity. This paper tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters in flash memory, but bringing them on demand to DRAM. Our method involves constructing an inference cost model that takes into account the characteristics of flash memory, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Within this hardware-informed framework, we introduce two principal techniques. First, "windowing" strategically reduces data transfer by reusing previously activated neurons, and second, "row-column bundling", tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory. These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.
Can LLMs Be Computers? | Percepta
We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.
Can LLMs Be Computers? | Percepta
We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.
i can’t help but think we’re far from nailing memory systems this one here is extremely interesting. two LLMs at once, one just managing and surfacing memory for the other
Asa ⭔
I'm not a fan of the decoupled 'memory retrieval → task execution' loop, so my agent has a subconscious background thread that looks for relevant, unique memory context in its experiential database while it runs and injects it on top of the live context window.
