ABSTRACT The relevance problem is the problem of how cognitive systems can be sensitive to what is relevant despite relevance being holistic, open‐ended, and context‐sensitive. The problem has typically been applied to higher‐order cognitive processes. In this paper, I argue that there is a hitherto unrecognized relevance problem for perceptual attention. This problem becomes apparent when considering the functions of attention: attention must not only support ongoing task performance but also remain sensitive to opportunities for task switching, latent learning, and context checking. Existing treatments of the relevance problem have largely focused on its task‐relative form: given a task, what is relevant? I argue that attention faces a more general problem that includes sensitivity to factors relevant to other interests and to cues indicating that another activity may be worth pursuing. For such features of the environment to stand out as salient, salience cannot be determined solely by bottom‐up factors and a limited set of occurrent top‐down factors, such as current goals and intentions. Rather, it must reflect the wider (if not the entire) range of an agent's latent interests in a context‐sensitive way. This motivational unencapsulation of attention poses a challenge for both contemporary accounts of attention and suggests that passive attention is much more intelligent than commonly thought.
Attention and the Relevance Problem
ABSTRACT The relevance problem is the problem of how cognitive systems can be sensitive to what is relevant despite relevance being holistic, open‐ended, and context‐sensitive. The problem has typically been applied to higher‐order cognitive processes. In this paper, I argue that there is a hitherto unrecognized relevance problem for perceptual attention. This problem becomes apparent when considering the functions of attention: attention must not only support ongoing task performance but also remain sensitive to opportunities for task switching, latent learning, and context checking. Existing treatments of the relevance problem have largely focused on its task‐relative form: given a task, what is relevant? I argue that attention faces a more general problem that includes sensitivity to factors relevant to other interests and to cues indicating that another activity may be worth pursuing. For such features of the environment to stand out as salient, salience cannot be determined solely by bottom‐up factors and a limited set of occurrent top‐down factors, such as current goals and intentions. Rather, it must reflect the wider (if not the entire) range of an agent's latent interests in a context‐sensitive way. This motivational unencapsulation of attention poses a challenge for both contemporary accounts of attention and suggests that passive attention is much more intelligent than commonly thought.
The attention market—and what is wrong with it
Attention is described as a “scarce commodity” that is traded in “a marketplace.” This, it is further claimed, contributes to a “widespread sense of attentional crisis.” But is there really an attention market, and if so, what, if anything, is wrong with it? We defend the claim that there are markets in attention. We provide an account of such attention markets and use that account to address what is morally wrong with them. Our account draws on knowledge of how attention works and what roles it plays in the mind. The attention market trades in an ability to influence our attention – somewhat (though not exactly) like the labor market trades in an ability to influence how we use our capacity for work. Specifically, the commodity it trades in is attentional landscaping potential, viz. the ability to systematically influence patterns of attention by changes to the sensory environment individuals are exposed to. Attention markets thus, we argue, commodify influence over a human capacity that plays a central role in shaping individual experience, agency, and belief formation. This feature of attention markets makes them ethically problematic. As markets in access to external influence, attention markets pose a special threat to individual autonomy and escape the classical liberal defense of free markets. Those who value autonomy should worry about the attention markets that exist today.
How We Pay Attention Changes the Very Shape of Our Brains
Imagine arriving at the airport just in time to catch a plane. Everything in your behavior betrays the heightened concentration of your attention. Your mind on alert, you look for the departures si…
Active Inference as Context Acquisition for AI Agents
Interactive AI agents must acquire the right context as efficiently as possible. When a user omits a constraint, preference, file, or task variable, an agent can proceed with a default assumption or spend tokens on a clarifying question, retrieval call, tool call, or prompt trial. We formulate this tradeoff as active inference for context acquisition. An inner inference step updates beliefs over a latent task state, and an outer decision selects the next context action, task action, or stop action to minimize expected free energy under cost. In deterministic settings, the epistemic term reduces to expected information gain, optionally normalized by token cost. We instantiate the framework in Optimal Question Asking (OQA), with exact posteriors and a dynamic programming oracle, and benchmark frontier language models on binary and multiway categorical tasks from 25 to 300 candidates. We also study clarification before generation and automated prompt optimization under token budgets. The formulation is model-agnostic and views active inference as a design principle for the context-acquisition layer of AI agents.
Overcoming information overload with circular attention economies - Cosmik Labs
Deficient executive control in transformer attention
Abstract Although transformers in large language models (LLMs) effectively implement a self-attention mechanism that has revolutionized natural language processing, they lack an explicit architecture for the executive control of attention found in humans, which is essential for resolving conflicts and selecting relevant information in the presence of competing computations and is critical for adaptive behavior. To investigate the impact of this limitation in LLMs, we employed the classic color Stroop task, widely regarded as the gold standard, to test the executive control of attention in these models. Our results revealed a typical conflict effect of underperformance in terms of accuracy in the incongruent condition (e.g. naming the color of the word RED in blue) compared with the congruent condition (e.g. naming the color of the word RED in red), in short word lists, similar to human performance. However, as the length of the word lists increased, performance on the incongruent condition degraded toward near-total performance collapse, even as accuracy in the congruent condition remained excellent, and word reading (e.g. reading the word RED [in red] or RED [in blue], ignoring the color) was near-perfect. These findings demonstrate that transformer attention mechanisms are fundamentally limited in their capacity for conflict resolution across extended contexts, and a failure to up-regulate control adaptively under rising interference. We suggest that incorporating executive control mechanisms akin to those in biological attention is crucial for achieving artificial general intelligence.
Attention (machine learning)
In machine learning, attention is a method that determines the importance of each component in a sequence relative to the other components in that sequence. In natural language processing, importance is represented by "soft" weights assigned to each word in a sentence. More generally, attention encodes vectors called token embeddings across a fixed-width sequence that can range from tens to millions of tokens in size.
Social-Information Seeking in Development: The Child as Experimental Psychologist
Research has established that children are “naive psychologists,” adept at understanding and navigating the social world from an early age. However, most of this work has focused on how children process information that they acquire incidentally, for example, by passively observing others’ actions. Here, we draw on literature framing children as intuitive scientists who actively seek information and test hypotheses to propose a view of children as naive experimental psychologists. From this perspective, children play an active role in selecting and pursuing relevant social information (e.g., about agents’ goals, traits, or relationships), whereby their search strategies are influenced both by context and task demands, as well as their prior beliefs, concepts, and domain-specific naive theories. We argue that the particular challenges associated with learning and reasoning about other minds may necessitate that children leverage their active learning competences, and we outline how the social domain uniquely constrains and shapes the learning process. We review existing research on social-information seeking in children and adults and identify directions for future research, emphasizing that children’s developing social cognition should be understood in terms of the active, exploratory role they take in learning about and participating in the social world.
Almost anything you give sustained attention to will begin to loop on itself and bloom
When people talk about the value of paying attention and slowing down, they often make it sound prudish and monk-like. But we shouldn’t forget how interesting and overpoweringly pleasurable sustained attention can be.
Social-information seeking in development: The child as experimental psychologist
Research has established that children are “naive psychologists”, adept at understanding and navigating the social world from an early age. However, most of this work has focused on how children process information that they acquire incidentally, for example by passively observing others’ actions. Here, we draw on literature framing children as intuitive scientists, who actively seek information and test hypotheses, to propose a view of children as naive experimental psychologists. From this perspective, children play an active role in selecting and pursuing relevant social information (e.g., about agents’ goals, traits, or relationships), whereby their search strategies are influenced both by context and task demands, as well as their prior beliefs, concepts, and domain-specific naive theories. We argue that the particular challenges associated with learning and reasoning about other minds may necessitate that children leverage their active learning competences, and we outline how the social domain uniquely constrains and shapes the learning process. We review existing research on social-information seeking in children and adults, and identify directions for future research, emphasizing that children’s developing social cognition should be understood in terms of the active, exploratory role they take in learning about and participating in the social world.
The Inversion Problem: Why Algorithms Should Infer Mental State and Not Just Predict Behavior
More and more machine learning is applied to human behavior. Increasingly these algorithms suffer from a hidden—but serious—problem. It arises because they often predict one thing while hoping for another. Take a recommender system: It predicts clicks but hopes to identify preferences. Or take an algorithm that automates a radiologist: It predicts in-the-moment diagnoses while hoping to identify their reflective judgments. Psychology shows us the gaps between the objectives of such prediction tasks and the goals we hope to achieve: People can click mindlessly; experts can get tired and make systematic errors. We argue such situations are ubiquitous and call them “inversion problems”: The real goal requires understanding a mental state that is not directly measured in behavioral data but must instead be inverted from the behavior. Identifying and solving these problems require new tools that draw on both behavioral and computational science.
Rethinking the cognitive foundations of the attention economy
The attention economy is the economic system in which human attention is the scarce resource. This literature takes a particular passage in Herbert Simon’s seminal Chapter Designing Organization fo...
Mindful Judgment and Decision Making
A full range of psychological processes has been put into play to explain judgment and choice phenomena. Complementing work on attention, information integration, and learning, decision research over the past 10 years has also examined the effects of goals, mental representation, and memory processes. In addition to deliberative processes, automatic processes have gotten closer attention, and the emotions revolution has put affective processes on a footing equal to cognitive ones. Psychological process models provide natural predictions about individual differences and lifespan changes and integrate across judgment and decision making (JDM) phenomena. “Mindful” JDM research leverages our knowledge about psychological processes into causal explanations for important judgment and choice regularities, emphasizing the adaptive use of an abundance of processing alternatives. Such explanations supplement and support existing mathematical descriptions of phenomena such as loss aversion or hyperbolic discounting. Unlike such descriptions, they also provide entry points for interventions designed to help people overcome judgments or choices considered undesirable.
Metacognitive Myopia: A Major Obstacle on the Way to Rationality
The notion of metacognitive myopia refers to a conspicuous weakness of the quality control of memory and reasoning processes. Although people are often remarkably sensitive even to complex samples of information when making evaluative judgments and decisions, their uncritical and naive tendency to take the validity of sampled information for granted constitutes a major obstacle to rational behavior. After illustrating this phenomenon with reference to prominent biases (base-rate neglect, misattribution, perseverance), we decompose metacognitive myopia into two distinct but intertwined functions, monitoring and control. We offer explanations for why effectively monitoring the biases resulting from information sampling in an uncertain world is so difficult and why the control function is severely restricted by the lack of volitional control over mental actions. Because of these and other difficulties, metacognitive myopia constitutes a major obstacle to rational judgment and decision making.
Relevant answers to polar questions
People often provide answers that go beyond what a question literally asks, but it has been difficult to pin down what makes some answers more relevant than others. Here, we introduce Pragmatic Reasoning In Overinformative Responses to Polar Questions (PRIOR-PQ), a probabilistic cognitive model formalizing how people use theory of mind (ToM) to produce and interpret relevantly overinformative answers to yes–no questions. Specifically, PRIOR-PQ grounds the pragmatics of question answering in inferences about the underlying goal that motivated the questioner to ask the given question as opposed to a different question. We evaluate our probabilistic model against human answering behaviour elicited in three case studies of increasing complexity, demonstrating its ability to predict nuanced patterns of relevance better than existing models, including state-of-the-art large language models. We also show how the goal-sensitive reasoning instantiated in our probabilistic model motivates a novel chain-of-thought prompting method allowing language models to approach more human-like performance. This work illuminates the mechanistic role of ToM in the pragmatics of question–answer exchanges, bridging formal semantics, cognitive science and artificial intelligence. Our findings have implications for developing more socially grounded dialogue systems and highlight the importance of integrating explanatory cognitive models with machine learning approaches. This article is part of the theme issue ‘At the heart of human communication: new views on the complex relationship between pragmatics and Theory of Mind’.
Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
Large language model (LLM) applications such as agents and domain-specific reasoning increasingly rely on context adaptation: modifying inputs with instructions, strategies, or evidence, rather than weight updates. Prior approaches improve usability but often suffer from brevity bias, which drops domain insights for concise summaries, and from context collapse, where iterative rewriting erodes details over time. We introduce ACE (Agentic Context Engineering), a framework that treats contexts as evolving playbooks that accumulate, refine, and organize strategies through a modular process of generation, reflection, and curation. ACE prevents collapse with structured, incremental updates that preserve detailed knowledge and scale with long-context models. Across agent and domain-specific benchmarks, ACE optimizes contexts both offline (e.g., system prompts) and online (e.g., agent memory), consistently outperforming strong baselines: +10.6% on agents and +8.6% on finance, while significantly reducing adaptation latency and rollout cost. Notably, ACE could adapt effectively without labeled supervision and instead by leveraging natural execution feedback. On the AppWorld leaderboard, ACE matches the top-ranked production-level agent on the overall average and surpasses it on the harder test-challenge split, despite using a smaller open-source model. These results show that comprehensive, evolving contexts enable scalable, efficient, and self-improving LLM systems with low overhead.