The Risk Reward Matrix helps you to balance Risk and Reward when choosing among options.
WRKSHP.tools | Uncertainty Matrix
The Uncertainty Matrix helps you find threats and opportunities that are uncertain but can have a big impact on your business
Risk of bias tools - Current version of RoB 2
Current version Download the 22 August 2019 version: The full guidance document. The cribsheet summarizing the tool. A template for completing the assessment. An Excel tool to implement RoB 2 (contains macros; download to your computer before using; some text is slightly out of date). We have
WRKSHP.tools | Riskiest Assumption Canvas
How do you know you’re making the right bet with your idea? Which bets does the success of your idea hinge on? These are your riskiest assumptions; they need to be tested.
Reward is not the optimization target — LessWrong
TurnTrout discusses a common misconception in reinforcement learning: that reward is the optimization target of trained agents. He argues reward is b…
LifeBench: Evaluating Large Language Models as Lifelong Decision-Makers in a Simulated Human Life
Large language models have improved rapidly on reasoning, coding, and mathematics, yet comparatively little is known about how these gains transfer to tasks that demand long-horizon planning, awareness of compounding consequences, and the management of risk and uncertainty. We introduce **LifeBench**, an evaluation suite that probes these capabilities by casting an LLM as the decision-maker in a simulated human life. LifeBench is a life-course harness: beginning at age 18, an agent commits a single action each year until death, spanning education, career, finance, health, relationships, and well-being. Each action modifies the underlying stocks that govern how the life trajectory evolves. The simulator is stochastic, and the evaluation is designed to reflect the uncertainty inherent to real lives. Life trajectories are scored on three fronts: wealth, health, and well-being. We aggregate these three fronts into a single index called the Life Trajectory Index (LTI). On this index, the evaluated models cluster in a narrow band, and all fall well short of a ceiling established by search over the action space. To resolve this clustering and to assess trajectories on more subjective grounds, we also score using an LLM-as-judge with research-grounded rubrics that compare lives pairwise, switching presentation order to reduce position bias and aggregating verdicts into a Bradley-Terry score per model. We further characterize the steering behavior these models exhibit over long horizons by ablating across prompt variants.
Cognitive Bias Lab | Learn to Make Better Decisions
Explore cognitive biases with interactive tests, simulations, and real-world examples. Free platform to sharpen decision-making and critical thinking — no sign-up needed.
WRKSHP.tools | Product Box
The Product Box exercise helps you when you need to convert an abstract concept into something tangible that you can evaluate with people that were not intimately part of the design process.
Reinforcement Learning from Human Feedback
The authoritative guide for Reinforcement learning from human feedback, alignment, and post-training LLMs. Aligning AI models to human preferences helps them become safer, smarter, easier to use, and tuned to the exact style the creator desires. Reinforcement Learning From Human Feedback (RHLF) is the process for using human responses to a model’s output to shape its alignment, and therefore its behavior. In Reinforcement Learning from Human Feedback, author Nathan Lambert blends diverse perspectives from fields like philosophy and economics with the core mathematics and computer science of RLHF to provide a practical guide you can use to apply RLHF to your models. In Reinforcement Learning from Human Feedback you’ll discover: How today’s most advanced AI models are taught from human feedback How large-scale preference data is collected and how to improve your data pipelines A comprehensive overview with derivations and implementations for the core policy-gradient methods used to train AI models with reinforcement learning (RL) Direct Preference Optimization (DPO), direct alignment algorithms, and simpler methods for preference finetuning How RLHF methods led to the current reinforcement learning from verifiable rewards (RLVR) renaissance Tricks used in industry to round out models, from product, character or personality training, AI feedback, and more How to approach evaluation and how evaluation has changed over the years Standard recipes for post-training combining more methods like instruction tuning with RLHF Behind-the-scenes stories from building open models like Llama-Instruct, Zephyr, Olmo, and Tülu After ChatGPT used RLHF to become production-ready, this foundational technique exploded in popularity. In Reinforcement Learning from Human Feedback, AI expert Nathan Lambert gives a true industry insider's perspective on modern RLHF training pipelines, and their trade-offs. Using hands-on experiments and mini-implementations, Nathan clearly and concisely introduces the alignment techniques that can transform a generic base model into a human-friendly tool.
Contracting for LLM Delegation: Moral Hazard in Technology and Effort Choice
We extend the standard Principal-Agent framework to scenarios where the Agent selects from a suite of technologies, each characterized by a distinct cost-capability profile. This framework is increasingly critical in the era of Large Language Models (LLMs), where Agents choose both a model and an associated effort level (e.g., token budget). We model the relationship between output quality and effort as a concave, saturating function, which depends on the Agent's hidden two-dimensional action choice balancing technology selection and effort allocation. We derive the optimal linear contract for the Principal, demonstrating that the Agent's best response is characterized by a threshold reward share that triggers technology switching. Finally, we calibrate our model using open-weight LLM pairings across the MATH and MMLUPro benchmarks. We show that both Principal and Agent, when employing bandit algorithms to navigate this environment, converge to strategies that closely align with our theoretical equilibrium. These results suggest that simple linear contracts can effectively incentivize complex, technology-aware delegation in agentic workflows.
Predicting outcomes for games of skill by redefining what it means to win
The Elo rating system is a highly successful ranking algorithm for games of skill where, by construction, one team wins and the other loses. A primary limitation of the original Elo algorithm is its inability to predict information beyond a match's win-loss probability. Specifically, the victor is awarded the same point bounty if he beats a team by 1 point or 10 points; only the rating difference between the team and its opponent affects the match bounty. In this work, we explain that Elo ratings and predictions can be naturally extended to include margin-of-victory information by simply redefining "what it means to win." We create ratings for each value of the margin-of-victory and use these ratings to predict the full distribution of point spread outcomes for matches which have not yet been played.
The AI Preference Cascade Reaches Farther
Those who know keep warning us about AI risks.
#Exploration: A Study of Count-Based Exploration for Deep...
Count-based exploration algorithms are known to perform near-optimally when used in conjunction with tabular reinforcement learning (RL) methods for solving small discrete Markov decision...
Weighted DVF: A Simple Model for Scoring and Prioritizing Ideas
Discover a powerful yet straightforward approach to evaluating new ideas by balancing desirability, viability, and feasibility. Learn how…
What is system-level investing?
System-level investing explicitly focuses on how relationships within and between systems impact investment risks and returns
Adult age differences in monetary decisions with real and hypothetical reward
Abstract Age differences in monetary decisions may emerge because younger and older adults perceive the value of outcomes differently. Yet, age‐differential effects of monetary rewards on decisions are not well understood. Most laboratory studies on aging and decision making have used scenarios in which rewards were merely hypothetical (decisions did not have any real consequences) or in which only small amounts of money were at stake. In the current study, we compared younger adults' (20–29 years) and older adults' (61–82 years) decisions in probabilistic choice problems with real or hypothetical rewards. Decision‐contingent rewards were in a typical range of previous studies (gains of up to ~4.25 USD) or substantially scaled up (gains of up to ~85 USD per participant). Reward type (real vs. hypothetical) affected decision quality, including value maximization, switching between options, and dominance violations (choices of an option that was inferior to another option in all respects). Decision quality was markedly better with real than hypothetical rewards in older adults and correlated with numeracy in both age groups. However, we found no evidence that reward type affected people's risk preferences. Overall, the findings portray a fairly positive picture regarding the use of hypothetical scenarios to assess preferences: With carefully prepared instructions, people from different age groups indicate preferences in hypothetical scenarios that match their decisions with real and much higher rewards. One advantage of using real rewards is that they help to reduce decision noise.
