Collection
Reading List
Updated 23d
An ephemeral list of things to read.
Collection
Updated 23d
An ephemeral list of things to read.
Cognitive Architectures for Language Agents
Recent efforts have augmented large language models (LLMs) with external resources (e.g., the Internet) or internal control flows (e.g., prompt chaining) for tasks requiring grounding or reasoning, leading to a new class of language agents. While these agents have achieved substantial empirical success, we lack a framework to organize existing agents and plan future developments. In this paper, we draw on the rich history of cognitive science and symbolic artificial intelligence to propose Cognitive Architectures for Language Agents (CoALA). CoALA describes a language agent with modular memory components, a structured action space to interact with internal memory and external environments, and a generalized decision-making process to choose actions. We use CoALA to retrospectively survey and organize a large body of recent work, and prospectively identify actionable directions towards more capable agents. Taken together, CoALA contextualizes today’s language agents within the broader history of AI and outlines a path towards language-based general intelligence.
Project vs. Product, Startup vs. Business
A couple of weeks ago I dropped into a thread by Mosh Lee, about how ATproto has too many projects and too few businesses. Yes! A few months ago I volunteered to help teams with ops or strategy, and k
Cognitive Surrender
Cognitive offloading is delegating to the AI and still owning the answer. Cognitive surrender is when the AI's output quietly becomes your output and there i...
How to Serve Local LLMs Anywhere: Secure Remote Access with Cloudflare and Unsloth | Unsloth Documentation
Unsloth is an open-source project that allows you to train and run LLMs locally and with Cloudflare tunnel, you can access Unsloth from your mobile device, share access to a friend or coworker, host Unsloth on a server such as Google Colab, AWS, or even a personal server.
Making Documentation Simpler and Practical: Our Docs-as-Code Journey — Squarespace Engineering Blog
In the fast-paced world of software development, documentation often gets a bad rap. It's perceived as a chore, a necessary evil, and sometimes, unfortunately, an afterthought. But what if writing documentation could be as dynamic and collaborative as writing the code itself? What if it could be si
Better Models: Worse Tools
About an aggravating tool-calling regression in newer Claude models.
Warnings for pattern matching
We examine the ML pattern-matching anomalies of useless clauses and non-exhaustive matches. We state the definition of these anomalies, building upon pattern matching semantics, and propose a simple algorithm to detect them. We have integrated the algorithm in the Objective Caml compiler, but we show that the same algorithm is also usable in a non-strict language such as Haskell. Or-patterns are considered for both strict and non-strict languages.
Open tabs are cognitive spaces
Nobody actually wants to use a web browser. It becomes evident the second you open your browser and notice that you don’t have internet connection. What we like about browsers is what they offer on the other side. What they connect us to. In that sense, browsers are transitory spaces - similar to airports, planes and train stations. Most of us have lost our childish curiosity towards these spaces - we go there because we want to arrive somewhere else. Arriving somewhere else with early browsers was quite one-dimensional - you clicked your way from one website to the next, in the same window. You surfed. You bookmarked the pages that you wanted to return to. Later, as the web became richer, you sometimes had more than one browser window open. I vividly remember having seven Internet Explorer windows open, thinking to myself “this is way too much to keep track of!”. Right now, my Firefox has 477 tabs open. How could this happen? Tabs are externalized cognition
An Interactive Intro to CRDTs | jakelazaroff.com
CRDTs don't have to be all academic papers and math jargon. Learn what CRDTs are and how they work through interactive visualizations and code samples.
Hope in Common
We seem to have reached an impasse. Capitalism as we know it appears to be coming apart. But as financial institutions stagger and crumble, there is no obvious alternative. Organized resistance appears scattered and incoherent; the global justice movement a shadow of its former self. There is good reason to believe that, in a generation […]
Project Glasswing: what Mythos showed us
In recent weeks, we pointed Mythos and other security-focused LLMs at live code across critical parts of our infrastructure. We share what we observed, the models’ strengths and weaknesses, and what the work around them needs to look like before any of it can scale.
Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches
Recent advancements in large language models (LLMs) have substantially improved automated code generation. While function-level and file-level generation have achieved promising results, real-world software development typically requires reasoning across entire repositories. This gives rise to the challenging task of Repository-Level Code Generation (RLCG), where models must capture long-range dependencies, ensure global semantic consistency, and generate coherent code spanning multiple files or modules. To address these challenges, Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm that integrates external retrieval mechanisms with LLMs, enhancing context-awareness and scalability. In this survey, we provide a comprehensive review of research on Retrieval-Augmented Code Generation (RACG), with an emphasis on repository-level approaches. We categorize existing work along several dimensions, including generation strategies, retrieval modalities, model architectures, training paradigms, and evaluation protocols. Furthermore, we summarize widely used datasets and benchmarks, analyze current limitations, and outline key challenges and opportunities for future research. Our goal is to establish a unified analytical framework for understanding this rapidly evolving field and to inspire continued progress in AI-powered software engineering.
Enhancing repository-level code completion with selective retrieval
Self-supervised method for learning when to retrieve contextual information from a code repository speeds up code completion times by 70% while increasing accuracy.
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
The task of repository-level code completion is to continue writing the unfinished code based on a broader context of the repository. While for automated code completion tools, it is difficult to utilize the useful information scattered in different files. We propose RepoCoder, a simple, generic, and effective framework to address the challenge. It streamlines the repository-level code completion process by incorporating a similarity-based retriever and a pre-trained code language model in an iterative retrieval-generation pipeline. RepoCoder makes effective utilization of repository-level information for code completion and has the ability to generate code at various levels of granularity. Moreover, we propose a new benchmark RepoBench, which consists of the latest and high-quality real-world repositories covering line, API invocation, and function body completion scenarios. Experimental results indicate that RepoCoder significantly improves the In-File completion baseline by over 10% in all settings and consistently outperforms the vanilla retrieval-augmented code completion approach. Furthermore, we validate the effectiveness of RepoCoder through comprehensive analysis, providing valuable insights for future research. Our source code and benchmark will be publicly available after the paper review.
Lost in the Middle: How Language Models Use Long Contexts
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
TDOP Pratt
Work reported herein was supported in part at Stanford by the National Science Foundation under grant no GJ 992, and the Office of Naval Research under grant number N-00014-67-A-Oll2-0057 NR 044-402; by IBM under a post-doctoral fellowship at Stanford; by the IBM T.J. Watson Research Center, Yorktown Heights, N.Y.; and by Project MAC, an MIT research program sponsored by the Advanced Research Projects Agency, Department of Defense, under Office of Naval Research Contract Number N000l4-70-O362-0006 and the National Science Foundation under contract number GJO0-4327. Reproduction in whole or in part is permitted for any purpose of the United States Government.
Trauma Junkie
[First place winner in the Synapse Storytelling Contest for Fiction.] It’s Christmastime, and over the hills, in the faraway furrows of one of those endless tracts of half-submerged land, a main battle tank sits in a ditch, decomposing, festering. My tank. But the neighborhood seems to have forgotten, here in the shadow of the old Hospital, that shining ziggurat high above. An aspiration, once — and an obligation.