Softmax forever, or why I like softmax

Leshem (Legend) Choshen 🤖🤗 @ACL @ICML on Twitter / X

I agree, what I say is that you talk about how much you lose of the gradient you compute, and I say, but if you just compute less, the result would lose less and still be equivalent or worse probably?More steps=more information that's true, but a bit of a separate battle right?— Leshem (Legend) Choshen 🤖🤗 @ACL @ICML (@LChoshen) August 17, 2026

Various LLM smells

Looks like this ended up on the HN front-page: HN Thread Late last year I started writing a math blog and decided to use LLMs to polish/enhance my writin...

Faulty reward functions in the wild

Reinforcement learning algorithms can break in surprising, counterintuitive ways. In this post we’ll explore one failure mode, which is where you misspecify your reward function.

Some notes on the Stable Diffusion safety filter

How deep learning models are moderated

Bayesian Influence Functions for Hessian-Free Data Attribution

Classical influence functions face significant challenges when applied to deep neural networks, primarily due to non-invertible Hessians and high-dimensional parameter spaces. We propose the local...

My Honest Review of Math Academy (Including Their Machine Learning Math Course)

Learning Math For Machine Learning on Math Academy

Derivatives of Spherical Harmonics

Christoph Peters. 2025–06 in High-Performance Graphics 2025. Poster.

Maximillian Piras on Twitter / X

Not sure when my @aiDotEngineer talk will be posted, so for those who asked here's a link to my slides (couldn't add notes so if you anything needs explanation just reply here & I'll thread the context): https://t.co/O1yg63iR3D https://t.co/yerleRwzHp pic.twitter.com/WTK82rPmDG— Maximillian Piras (@MVXMXM) June 14, 2025

tinygrad: A simple and powerful neural network framework

We write and maintain tinygrad, the fastest growing neural network framework

Influence Functions in Deep Learning Are Fragile

Influence functions approximate the effect of training samples in test-time predictions and have a wide variety of applications in machine learning interpretability and uncertainty estimation. A commonly-used (first-order) influence function can be implemented efficiently as a post-hoc method requiring access only to the gradients and Hessian of the model. For linear models, influence functions are well-defined due to the convexity of the underlying loss function and are generally accurate even across difficult settings where model changes are fairly large such as estimating group influences. Influence functions, however, are not well-understood in the context of deep learning with non-convex loss functions. In this paper, we provide a comprehensive and large-scale empirical study of successes and failures of influence functions in neural network models trained on datasets such as Iris, MNIST, CIFAR-10 and ImageNet. Through our extensive experiments, we show that the network architecture, its depth and width, as well as the extent of model parameterization and regularization techniques have strong effects in the accuracy of influence functions. In particular, we find that (i) influence estimates are fairly accurate for shallow networks, while for deeper networks the estimates are often erroneous; (ii) for certain network architectures and datasets, training with weight-decay regularization is important to get high-quality influence estimates; and (iii) the accuracy of influence estimates can vary significantly depending on the examined test points. These results suggest that in general influence functions in deep learning are fragile and call for developing improved influence estimation methods to mitigate these issues in non-convex setups.

Donald MacKenzie · AI’s Scale

AI’s scale doesn’t matter just to specialists. The rest of us are being taken on a ride along the logarithmic curve...

Maximillian Piras Talk Updates | Yutori

Tell me when Maximillian Piras’ talk The Bitter Layout gets posted to the AI Engineer YouTube channel or any other relevant news or mentions of his talk

If Influence Functions are the Answer, Then What is the Question?

Influence functions efficiently estimate the effect of removing a single training data point on a model's learned parameters. While influence estimates align well with leave-one-out retraining for linear models, recent works have shown this alignment is often poor in neural networks. In this work, we investigate the specific factors that cause this discrepancy by decomposing it into five separate terms. We study the contributions of each term on a variety of architectures and datasets and how they vary with factors such as network width and training time. While practical influence function estimates may be a poor match to leave-one-out retraining for nonlinear networks, we show that they are often a good approximation to a different object we term the proximal Bregman response function (PBRF). Since the PBRF can still be used to answer many of the questions motivating influence functions, such as identifying influential or mislabeled examples, our results suggest that current algorithms for influence function estimation give more informative results than previous error analyses would suggest.

Tony Feng on Twitter / X

I am a mathematician but I avoided commenting on this because the problems are (far) outside my domain. Even if I sat down to read the technical details (which I have not), it would be hard to appreciate the context of prior work, etc. But I've gotten enough sense of things… https://t.co/VZzosBYFto— Tony Feng (@tonylfeng) August 5, 2026