The world is racing to develop ever more sophisticated large language models while a small language model unfurls itself in my home.
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Stochastic Parrots | Emily M. Bender - Professor of Computational Linguistics
Post by Paul Musgrave (@professormusgrave.bsky.social) on Bluesky — View on Aturi
I don't know, man, can a stochastic parrot take a half-remembered anecdote and systematically refine its internet searching to come up with *exactly* the resour
Are Emergent Abilities of Large Language Models a Mirage?
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities...
No, “AI” is not a Stochastic Parrot 🦜
I’ve recently come across a new flavor of AI denialism making the rounds.
The Unpredictable Abilities Emerging From Large AI Models | Quanta Magazine
Large language models like ChatGPT are now big enough that they’ve started to display startling, unpredictable behaviors.
LLMs are good at playing you
Large language models seem just like humans, but some of this is in our heads.
Polly Wants a Better Argument
The “Stochastic Parrot” Argument is Both Wrong and Actively Harmful
The evolution of syntactic communication
Animal communication is typically non-syntactic, which means that signals refer to whole situations1,2,3,4,5,6,7. Human language is syntactic, and signals consist of discrete components that have their own meaning8. Syntax is a prerequisite for taking advantage of combinatorics, that is, “making infinite use of finite means”9,10,11. The vast expressive power of human language would be impossible without syntax, and the transition from non-syntactic to syntactic communication was an essential step in the evolution of human language12,13,14,15,16. We aim to understand the evolutionary dynamics of this transition and to analyse how natural selection can guide it. Here we present a model for the population dynamics of language evolution, define the basic reproductive ratio of words and calculate the maximum size of a lexicon. Syntax allows larger repertoires and the possibility to formulate messages that have not been learned beforehand. Nevertheless, according to our model natural selection can only favour the emergence of syntax if the number of required signals exceeds a threshold value. This result might explain why only humans evolved syntactic communication and hence complex language.
AI learns language from skewed sources. That could change how we humans speak – and think | Bruce Schneier
Large language models aren’t trained on real-life conversations. As we encounter their language, it could affect our own
How Large Language Models Actually Work
Mapping the Mind of a Large Language Model
We have identified how millions of concepts are represented inside Claude Sonnet, one of our deployed large language models. This is the first ever detailed look inside a modern, production-grade large language model.
Hallucination is Inevitable: An Innate Limitation of Large Language Models
Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts...
Models overview
Claude is a family of state-of-the-art large language models developed by Anthropic. This guide introduces the available models and compares their performance.
I dunno how many times someone has told me, “the stochastic parrots paper was wrong,” and i’ve had to stutter, regain my composure, and put in real work to salvage the conversation. Like lol, wrong about what? That coherence is in the eye of the beholder? We see things as we are, not as they are.
Emily M. Bender
In this context, I feel like reminding people (again) that the stochastic parrots paper was not primarily a response to synthetic text extruding machines (not at all popular in late 2020), but an exploration of the range harms that had already been documented in the pursuit of LM scale. >>
