When the Scoreboard Becomes the Game, It’s Time to Recalibrate Research Metrics - The Scholarly Kitchen
Today's guest post discusses research metrics and their relationship to research integrity, inclusivity, and long-term impact.
Opinion | I Didn’t Want It to Be True, but the Medium Really Is the Message (Published 2022)
How we look matters as much as what we see.
On Taste, Effort & Curiosity - again
When AI collapses how long it takes to ship, what’s left is judgment, experimentation, and knowing what not to build.
Artificial intelligence and illusions of understanding in scientific research
Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists’ visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community’s ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.
Visualize Value — Ideas, made visible
An ongoing practice of turning ideas into images. Visual ideas about work, markets, technology, culture, and human behavior by Jack Butcher.
Deep research requires a slower pace than tech industry work
Anyone working in an industry for a while will become accustomed to that culture—its processes, its norms, its values, its tacit knowledge. Much of this is incredibly valuable, of course, but these ideas can also represent constraints. There are some important impedances here between tech industry culture and research culture. In particular, tech culture is calibrated to a much faster pace. This can lead to impatience or early abandonment when confronting problems which require a researcher’s pace.
Deep research requires a slower pace than tech industry work
Anyone working in an industry for a while will become accustomed to that culture—its processes, its norms, its values, its tacit knowledge. Much of this is incredibly valuable, of course, but these ideas can also represent constraints. There are some important impedances here between tech industry culture and research culture. In particular, tech culture is calibrated to a much faster pace. This can lead to impatience or early abandonment when confronting problems which require a researcher’s pace.
The Golden Age Is Behind Us: How the Status Quo Impacts the Evaluation of Technology
New technology invariably provokes concerns over potential societal impacts. Even as risks often fail to materialize, the fear continues. The current research explored the psychological underpinnings of this pattern. Across four studies ( N = 2,454 adults recruited via Amazon Mechanical Turk), we found evidence for the role of status quo thinking in evaluating technology. In Study 1, we experimentally manipulated the reported age of unfamiliar technology and found that people evaluate it more favorably when it is described as originating before (vs. after) their birth. In Studies 2 through 4, participants’ age at the time of invention strongly predicts attitudes toward a wide range of real-world technologies. Finally, we found that individual differences in status-quo-based decision-making moderated evaluations of technology. These studies provide insight into how people respond to the rapidly changing technological landscape.
AI and the Future of Science
A sampling model of social judgment.
Why Reading Matters - Cal Newport
Last week, Rose Horowitch published a splashy Atlantic article titled “The End of Reading is Here.” (Ironically, given the subject matter, it weighed in at ... Read more
Guide to Incrementals/Appeal to Developers | The Paper Pilot
The Paper Pilot's digital garden of thoughts on philosophy, politics, and sociology
Research – Open Heart Mind
We’re developing Metavaluation — a participatory framework for recognising and rewarding the value of diverse contributions, from creative work and care to logistics and leadership.
Expecting researcher taste and judgement to become increasingly valuable (this is from Anthropic's Opus 5.5 system card) anthropic.com/claude-opus-5-5-system-card
