Alignment Is Proven To Be Solvable

That LLMs understand natural language as well as they do should dramatically change our understanding of the problem.

Why Can’t Powerful LLMs Learn Multiplication?

These days, large language models (LLMs) can handle increasingly complex tasks, writing complex code and engaging in sophisticated reasoning. But when it comes to 4-digit multiplication, a task taught in elementary school, even state-of-the-art systems fail. Why? A new paper by Computer Science PhD student Xiaoyan Bai and Faculty Co-Director of the Data Science Institute’s …

LLMs and World Models, Part 1

How do Large Language Models Make Sense of Their “Worlds”?

Here’s what’s really going on inside an LLM’s neural network

Anthropic's conceptual mapping helps explain why LLMs behave the way they do.

Take caution in using LLMs as human surrogates | PNAS

Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, survey...

Illusions of Understanding from Outsourcing Thinking to LLMs

Some illusions of understanding are an inevitable part of the research process, while others can be avoided or overcome by careful critical thinking and observation. We are facing an increased risk of avoidable illusions as more research activities are delegated to large language models (LMM). LLMs can be useful but they cannot think, and their use can undermine our thinking and understanding. Thinking for ourselves is hard and error prone but worthwhile - and there are no shortcuts to understanding.

There's Something Fundamentally Wrong With LLMs

LLMs aren't trained on the "vast majority of speech," experts warn, a major blind spot that could have sweeping consequences.

What Happens, Exactly, When a Person Talks to an LLM?

A phenomenology of thinking with a model.

Understanding Understanding: A Pragmatic Framework Motivated by...

Motivated by the rapid ascent of Large Language Models (LLMs) and debates about the extent to which they possess human-level qualities, we propose a framework for testing whether any agent (be it...

The Return of Language-Oriented Programming | Middle of Nowhere

I’ve been wondering what LLMs mean for language design and implementation. Some believe that, because language models are obviously trained on existing content, they are inherently less capable of assisting users with new programming languages. Intuitively this makes sense. However:

How LLMs Actually Work

A from-the-ground-up walkthrough of how modern LLMs work, from tokens to transformer blocks to the next-token loop

Large language model

A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts, and are a foundational technology behind modern chatbots.[1] Biased or inaccurate training data can make an LLM's output less reliable.[2]

A Rational Analysis of the Effects of Sycophantic AI

People increasingly use large language models (LLMs) to explore ideas, gather information, and make sense of the world. In these interactions, they encounter agents that are overly agreeable. We...

Understanding Multimodal LLMs

An introduction to the main techniques and latest models

Why Are LLMs Smart?

A popular way to explain how current LLMs work is to say that “all” they do is predict the next most likely word in a sentence.