







In the summer, while I was working on a feature locally, I noticed a test started failing. I initially dismissed it: I assumed there was some flakiness, and CI was green, so everything must’ve been good, right? Until it kept failing, only locally. A test that a few days before was working as normal.
The Test That Only Failed in the Summer
In the summer, while I was working on a feature locally, I noticed a test started failing. I initially dismissed it: I assumed there was some flakiness, and CI was green, so everything must’ve been good, right? Until it kept failing, only locally. A test that a few days before was working as normal.

What Went Wrong with Coronavirus Testing in the U.S.
During three crucial weeks in February, as a first set of test kits sent out by the C.D.C. failed to work properly, labs across the country scrambled to fill the void.

Flaky Tests Are a Leadership Failure: Stop Apologizing
Stop blaming your scripts for flakiness. Learn why it is a sign of leadership failure and learn the 1st pillars of testability: Operability & Observability.

test highlights - underreacted
Testing can be fun, actually
Writing and maintaining tests is boring. But they're also some of the most valuable code we can write. With this blog post you'll learn a criminally underrated testing technique to add to your testing toolbox that can make tests a whole lot more pleasant.


Ephemeral testing
We have many ways to ensure software quality. Unit testing. Fuzz testing. Integration testing. And so forth. I'd like to propose a method that was unthinkable before: ephemeral testing. (Ephemeral is a fancy word for 'throw away' or 'temporary'.) You write your code. You build your software component. Or the AI agent does … Continue reading Ephemeral testing

Blood tests for mental health problems | Eiko Fried
A new paper was published yesterday on a blood test for schizophrenia, by the same research team that in 2021 published a paper on a blood test for depression . The papers and...

Half of social-science studies fail replication test in years-long project
Results from massive, ‘eagerly awaited’ initiative reinforce concerns about the credibility of science — but raise hope for solutions.

The Racist Beginnings of Standardized Testing | NEA
From grade school to college, students of color have suffered from the effects of biased testing.

Testing functional UIs - Hayleigh Thompson | Lambda Days 2025
The sad state of property-based testing libraries
Property-based testing is a rare example of academic research that has made it to the mainstream in less than 30 years. Under the slogan “don’t write tests, generate them” property-based testing has gained support from a diverse group of programming language communities. In fact, the Wikipedia page of the original property-basted testing Haskell library, QuickCheck, lists 57 reimplementations in other languages.
Failures count more than Successes
User-experience is defined by the times when a computer doesn't work, not by the times when it does.

Sophie Schmieg (@sophieschmieg@infosec.exchange)
All shitposting aside, though, if we go off the explanation the CA has given then some reformating script accidentally dropped most of the bytes or never requested them from the HSM or similar. A couple of years ago, we had a similar worry with Tink. In particular, we wanted to have a cheap test that catches obviously non-random values with high confidence. I added three little statistical test back then, which are run as unit test in every presubmit on everything that is supposed to look like a uniform random string. The tests are not constant time, so you cannot run them over your actual keys as is, but they are very reliable to test tooling that is supposed to handle and transform key material. They are tuned aggressively to detect non-randomness only up to 10 standard deviations, which makes the test extremely non flaky, at the cost of missing some events. But as I said, they're designed to be unit tests, that run with high frequency, so false positives would be very bad in that scenario. There are multiple ways to test this, some more principled than others, and somewhat depending on other requirements, but this is what I added to Tink back then: The first test is a z test that checks that every bit in the input string has a probability of 0.5 of being set. I.e. it sums all bits and checks whether the result is within 10 standard deviations of half of the length of the string. The second test is a cross correlation test that takes two bit strings, xors them, and runs the first test on the result. In other words it checks whether two independently drawn strings agree at half the positions. The third test is the most expensive one to run, but also the most powerful of these tests and tests the autocorrelation of a string. It rotates the string, shifting each bit by one and putting the last bit to the front, and then runs the cross correlation test on that string with the original string, for each possible shift. This test detects repetitions of sequences within a string, and will e.g. detect that a string contains only printable ASCII, although at the setting of 10 standard deviations it will require a couple of sentences before firing. https://github.com/tink-crypto/tink-cc/blob/3c94644e5b2dffacd66764da13083a97ee782d15/tink/util/test_util.h#L136