Notation as a Tool of Thought

J Notation as a Tool of Thought

Kenneth Iverson’s 1964 language, APL, won him the Turing Award. His award lecture, Notation as a Tool of Thought, argued that better notations would lead people to deeper insights about mathematics. He provided a number of examples ranging across linear algebra, arithmetic, probability, and logic. Unfortunately, most of the mathematics he covers isn’t relevant to programming. However, his core idea still applies, and changing how we describe programs changes how we think about them.

Semble

Semble is a legal term used when discussing published opinions. The word is the Norman (and Modern) French verbal form for[1] meaning "it seems or appears to be" [1] or, more simply, "it seems".[2][3]

Many Minds: Seven metaphors for AI

If you wanted a petri dish for understanding metaphors—how they emerge and evolve and jostle with each other—it would be hard to do better than the world of AI. We talk about AI systems variously as coaches or co-pilots, little genies or alien intelligences. Some researchers claim that AIs "grow," that they're entering their phase of "adolescence." Critics deride AI products as slop and dismiss LLMs as a kind of autocomplete on steroids. What's behind these different characterizations? Which ones are accurate and which are unfair? And are our metaphors mostly colorful rhetoric or do they matter? Are they shaping how we understand, adopt, and ultimately regulate these new technologies?   My guest today is . Melanie is a computer scientist and Professor at the Santa Fe Institute. She is the author of the book, and she writes a  by the same name.  This episode is a bit of a companion to  with Steve Flusberg. In that episode, Steve and I attempted a kind of crash course on metaphor and the human mind. Here, Melanie and I sit down for more of an extended case study: how metaphors are guiding, galvanizing, and maybe deceiving us in the contested realm of AI discourse. We unpack seven of the most widely used metaphors in this space. We consider how these metaphors are shaping not only our everyday understandings of AI, but also law and policy. We also talk about the metaphor and analogy capabilities of AI itself. Can these systems reason abstractly in the way that humans can? Along the way, Melanie and I touch on: AI-generated poetry, anthropomorphism, the original sin of AI research, the myth of Narcissus, psychometric testing and its pitfalls, metaphors for AI that are a bit hard to spot, and the question of whether an AI has ever come up with a decent analogy for itself.  Longtime fans of the show will know that we've had Melanie on the show . We invited her back, not only because she's thought about metaphor and analogy in AI discourse for decades, but because she's a voice of calm insight in an area that’s increasingly awash in hype and polemic. Longtime fans of the show may also note that we are now celebrating our 6th birthday at Many Minds. That's right, the show launched in February 2020. If you'd like to support us as we recognize this milestone, you can leave us a rating or a review, recommend us to a friend, or give us a shout out on social media. Your support is always appreciated.  Without further ado, on to my conversation with Dr. Melanie Mitchell. Enjoy!    Notes 3:30 – For an overview of Douglas Hofstadter’s work on analogy, see . 8:00 – Much of our discussion in this interview draws on Dr. Mitchell’s piece on the  in Science magazine.  13:30 – For earlier discussions of anthropomorphism on the show, see our earlier episodes and .  16:00 – See for the original discussion of LLMs as “stochastic parrots.” 17:00 –  See for the original discussion of ChatGPT as a “blurry jpeg.” 18:30 – See for the original discussion of LLMs as role players. 22:00 – See for one use of the “LLMs as crowds” metaphor. See also a discussion of this metaphor (and other metaphors for AI) .   25:00 – For one discussion of AI as a “cultural technology” by Alison Gopnik and colleagues, see . For a more recent discussion of the same metaphor by Henry Farrell, Alison Gopnik and others, see . 27:00 – For the podcast series on intelligence that Dr. Mitchell co-hosted for the Santa Fe Institute, see .  28:00 – See for an influential formulation of the idea that AI is an “alien intelligence.”  29:00 – For philosopher Shannon Vallor’s book about AI as “mirror,” see . 31:00 – For the recent study on users’ metaphors for AI systems, see .  33:00 – For more on the rise of social AI, see our earlier episode .  38:00 – For more on what AI researchers might learn from developmental and comparative psychologists, see Dr. Mitchell’s (summarizing her keynote at NeurIPs).  42:00 – For more on the ARC (Abstraction and Reasoning Corpus) and the research that Dr. Mitchell and colleagues have been doing with it, see and . 48:30 – For the study on humans' preference for AI-generated poetry, see . 50:30 – For Brigitte Nerlich’s documentation and discussion of various metaphors for AI (including AI’s metaphors for itself), see .   Recommendations  , by Shannon Vallor ‘,’ by Murray Shanahan (!) et al. ‘,’ by Henry Farrell et al.   Many Minds is a project of the , which is made possible by a generous grant from the John Templeton Foundation to Indiana University. The show is hosted and produced by , with help from Assistant Producer  and with creative support from DISI Directors Erica Cartmill and Jacob Foster. Our artwork is by . Subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you listen to podcasts. You can also now subscribe to the Many Minds newsletter ! We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com. For updates about the show, visit  or follow us on Bluesky ().

Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!

Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.

Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!

Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.

supernote-cli: pen, paper, and a pipe

A recent  NYT piece  argued we need a mental fitness revolution to combat the cognitive decay caused by algorithmic feeds and generative AI. It's an efficient one-two punch. If you're not brainrotting on short form video content, you're outsourcing all of your thinking to an LLM. The result is a kind of cognitive strip-mining. What's left requires active defense. For me, one way of defending that capacity for deep work is with a pen on e-ink. Whether it's annotating a paper or starting a sketch from scratch, I'm intentionally making room for focused thought. My army of clawed Claudes and Codexes will just have to wait.

an elegy for the pencil

how computers changed the way we write

Thinking modes

Floating in linguistic space.