Luminal

Benchmarking Subquadratic’s latest model & SSA Kernel | Appen

56× faster than FlashAttention-2 at 1M tokens. Independent efficiency, retrieval, and SWE-Bench benchmark of sparse self-attention. Download the full report.

Compiling Models to Megakernels

Fine-grained synchronization, deep pipelines, and zero kernel launch overheads, automatically.

jafioti/luminal | DeepWiki

This document provides a high-level introduction to the luminal deep learning framework, covering its core architecture, design principles, and key components. This overview explains how luminal's gra

Modern GPU Programming For MLSys — Modern GPU Programming For MLSys

Machine learning systems sit at the heart of modern AI workloads. In these systems, performance often comes down to the quality of a small number of GPU kernels. Attention kernels, LLM prefill and decode kernels, low-precision block-scaled GEMMs, fused MoE layers, and other large fused kernels all directly shape end-to-end speed in both training and serving.

Modern GPU Programming For MLSys — Modern GPU Programming For MLSys

Machine learning systems sit at the heart of modern AI workloads. In these systems, performance often comes down to the quality of a small number of GPU kernels. Attention kernels, LLM prefill and decode kernels, low-precision block-scaled GEMMs, fused MoE layers, and other large fused kernels all directly shape end-to-end speed in both training and serving.

Can LLMs Be Computers? | Percepta

We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.

Can LLMs Be Computers? | Percepta

We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.

Andrej Karpathy on Twitter / X

The race for LLM "cognitive core" - a few billion param model that maximally sacrifices encyclopedic knowledge for capability. It lives always-on and by default on every computer as the kernel of LLM personal computing.Its features are slowly crystalizing:- Natively multimodal… https://t.co/2jsVevkTSJ— Andrej Karpathy (@karpathy) June 27, 2025

Beyond Standard LLMs

Linear Attention Hybrids, Text Diffusion, Code World Models, and Small Recursive Transformers

Attention Is All You Need

"Attention Is All You Need" is a 2017 research paper in machine learning authored by eight scientists and engineers working at Google. The paper introduced a new deep learning architecture known as the transformer, based on the attention mechanism proposed in 2014 by Bahdanau et al. The transformer approach it describes has become the main architecture of a wide variety of artificial intelligence systems, including large language models. At the time, the focus of the research was on improving Seq2seq techniques for machine translation, but the authors go further in the paper, foreseeing the technique's potential for other tasks like question answering and what is now known as multimodal generative AI.

Introducing Lumo 1.1 for faster, advanced reasoning | Proton

Lumo 1.1 is a faster, smarter AI assistant that matches Big Tech’s capabilities while protecting your privacy with zero-access encryption.

A world of active objects for work and play: the first ten years of lively

The Lively Kernel is a complete platform for Web programming written in JavaScriptTM using graphics available in leading browsers. A widget set built from these elements provides a user interface kit, and the widget set is also extensible. A window-...

Create | Diffuse

You can generate interfaces or features easily using AI. Diffuse provides a skill to use with LLMs. Claude usage example:

Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli | Keith Doelling

This is some very cool work by some awesome colleagues Jean-Rémi King, and Teon Brooks! Seriously not enough good things can be said about how cool it is. You should enjoy it and play with it. And kudos to Meta for open sourcing it. At the same time, I'm already seeing posts about how the model will replace fMRI experiments as researchers will simulate how the brain "really works" instead of running costly experiments. I think this goes WELL beyond what its creators intend. We are already seeing that use of AI in science allows you to explore charted ideas more thoroughly and much more rapidly but slows us down in finding novel ideas (https://lnkd.in/eMR2akqt). At the same time, there is growing concern that LLM performance will collapse as they are increasingly trained on their own output (https://lnkd.in/eavgfyuY). Leaving neuroscience to AI simulations risks following the same fate, where we generate seemingly new findings without gaining new meaning. A mechanistic understanding of how the brain works (if that is still your goal) will be found at the margins, in errors and idiosyncrasies of neural function. What TRIBE provides is a super useful and cool instantiation of our current understanding on how and where neural activity is instantiated in the brain. But it won't help us make groundbreaking new findings of how neural circuits lead to cognition and behavior. Experiments on real human brains, may be costly, but they will always be necessary!