Why reasonable explanations are one of the easiest ways to avoid changing anything.
Nutshell: make expandable, embeddable explanations
a tool to let your readers dive into details
The Laws of Simplicity / John Maeda
The simplest way to achieve simplicity is through thoughtful reduction.
The Surprising Value of Obvious Insights | MIT Sloan Management Review
Confirming what people already believe can sometimes help organizations overcome barriers to change.
God Help Us, Let’s Try To Learn About Mechanistic Interpretability Techniques
...
Why Facts Don’t Change Our Minds
New discoveries about the human mind show the limitations of reason.
Resource-rational belief revision can mitigate as well as amplify polarization
People's beliefs sometimes diverge after observing the same information, which has been interpreted as evidence of irrationality. This behaviour has been proposed to result from people's limited cognitive resources and motivated reasoning, but how belief revision differs across these explanations has not been formalized or compared to a rational norm. Further, while people may be biased relative to a normative ideal, they may still make optimal choices given their limited cognitive resources, or rationally balance the utility of holding accurate beliefs with the belief's intrinsic utility. Across two studies, we develop and test a unified computational account of belief polarization under these proposed mechanisms, showing that people's performance on a belief updating task best fits a limited-resource Bayesian model; external motivations may contribute to divergence (or convergence) by determining what pre-existing information people consider relevant to a situation, rather than by changing how people evaluate new information in isolation.
Is AI Reasoning Right for the Wrong Reasons? | Quanta Magazine
The idea that artificial intelligence can “reason” is more intuitive than ever. But intuitions can be wrong, and the science is far from settled.
How To Think Real Good | Meta-rationality
Practical epistemology: heuristics for how to think about difficult things, particularly using formal methods.
fenc.es — be wrong on the internet, productively
Trace the map of reasonable disagreement. Break arguments into statements, rate confidence and importance, and find the crux.
Strange and maddening rules
There’s this popular idea among developers that when you face a problem with code, you should get out a rubber duck and explain, to the duck, exactly how your code was supposed to work.
Description not evaluation - The Cynefin Co
One of the things I am enjoying at the moment is the way a lot of things are coming together conceptually. It happens like this when you develop or repurpose knowledge from different sources. Individual practices and ideas make sense in their own right. Then you read more, practice more, and the various origins […]
Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
Large Language Models (LLMs) can achieve strong performance on many tasks by producing step-by-step reasoning before giving a final output, often referred to as chain-of-thought reasoning (CoT). It is tempting to interpret these CoT explanations as the LLM's process for solving a task. This level of transparency into LLMs' predictions would yield significant safety benefits. However, we find that CoT explanations can systematically misrepresent the true reason for a model's prediction. We demonstrate that CoT explanations can be heavily influenced by adding biasing features to model inputs--e.g., by reordering the multiple-choice options in a few-shot prompt to make the answer always "(A)"--which models systematically fail to mention in their explanations. When we bias models toward incorrect answers, they frequently generate CoT explanations rationalizing those answers. This causes accuracy to drop by as much as 36% on a suite of 13 tasks from BIG-Bench Hard, when testing with GPT-3.5 from OpenAI and Claude 1.0 from Anthropic. On a social-bias task, model explanations justify giving answers in line with stereotypes without mentioning the influence of these social biases. Our findings indicate that CoT explanations can be plausible yet misleading, which risks increasing our trust in LLMs without guaranteeing their safety. Building more transparent and explainable systems will require either improving CoT faithfulness through targeted efforts or abandoning CoT in favor of alternative methods.
when everyone has a different explanation of why the bad numbers are not that bad or why the small number is still very important, the simplest explanation is that the bad number is actually just indeed very bad
Christopher Mims
So I guess either Bluesky becomes a nonprofit or we all start sharing our Threads handles on here…?
