The Winning Variant

A/B testing, algorithmic optimization, and the small act of ignoring both — and what it means to make something by hand when a number can tell you what would have worked better.

Statistical Foundations of LLM-based A/B Testing: A Surrogacy...

Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in the hope of experimenting faster and at lower cost. We...

CLM - Another test claim

Some text

The new 20% time, minus the time

Attention is replacing hours. But is it 120% time all over again?

A universal approach to mocking

Defunctionalise your continuations and your tests can run any computation a step at a time.

Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing

SUMMARY The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of the false discovery rate rather than that of the FWER is desired, there is potential for a gain in power. A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics, and a simulation study shows that the gain in power is substantial. The use of the new procedure and the appropriateness of the criterion are illustrated with examples.

Blood tests for mental health problems | Eiko Fried

A new paper was published yesterday on a blood test for schizophrenia, by the same research team that in 2021 published a paper on a blood test for depression . The papers and...

Sᴜɴʀɪsᴇ Oᴀᴛʜ 十 Tʜᴇ Sʏʟʟᴀʙᴜꜱ (ORI) - CC0 on Twitter / X

It is significant that the passage of time involves the continuous transfer of data from one timestamp to another.— Sᴜɴʀɪsᴇ Oᴀᴛʜ 十 Tʜᴇ Sʏʟʟᴀʙᴜꜱ (ORI) - CC0 (@sunriseoath) September 23, 2026

Does Your Brain Think Accurately About Time?

Chicago Booth’s Kristin Donnelly talks about her research on time-duration asymmetry.

The control of the false discovery rate in multiple testing under dependency

Benjamini and Hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems. A simple procedure was given there as an FDR controlling procedure for independent test statistics and was shown to be much more powerful than comparable procedures which control the traditional familywise error rate. We prove that this same procedure also controls the false discovery rate when the test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses. This condition for positive dependency is general enough to cover many problems of practical interest, including the comparisons of many treatments with a single control, multivariate normal test statistics with positive correlation matrix and multivariate $t$. Furthermore, the test statistics may be discrete, and the tested hypotheses composite without posing special difficulties. For all other forms of dependency, a simple conservative modification of the procedure controls the false discovery rate. Thus the range of problems for which a procedure with proven FDR control can be offered is greatly increased.

Monday, October 5th

Each talk combined with QA is 45 minutes, followed by a 15 minute break.

Revisiting Joel's Test - exe.dev blog

Joel Spolsky's twelve questions still hold, but agents demand nine more. Call it the Shelley test.

Testing can be fun, actually

Writing and maintaining tests is boring. But they're also some of the most valuable code we can write. With this blog post you'll learn a criminally underrated testing technique to add to your testing toolbox that can make tests a whole lot more pleasant.