LLaDA - Large Language Diffusion Models

Continuous diffusion language models

Fully discrete methods dominated for a few years, but language models based on continuous diffusion are making a comeback.

Continuous diffusion language models

Fully discrete methods dominated for a few years, but language models based on continuous diffusion are making a comeback.

Research – Inception

We are leveraging diffusion technology to develop a new generation of LLMs. Our dLLMs are much faster and more efficient than traditional autoregressive LLMs.

Scaling Laws Across Model Architectures: A Comparative Analysis of...

The scaling of large language models (LLMs) is a critical research area for the efficiency and effectiveness of model training and deployment. Our work investigates the transferability and...

Alex Veremeyenko on Twitter / X

what are large language models actually doing?i read the 2025 textbook "Foundations of Large Language Models" by tong xiao and jingbo zhu and for the first time, i truly understood how they work.here’s everything you need to know about llms in 3 minutes↓ pic.twitter.com/OeYLuU3cWh— Alex Veremeyenko (@alex_verem) July 26, 2025

The Case Against LLMs as Rerankers

Authors: Apoorva Joshi, Zhenmei Shi, Akshay Goindani, Hong LiuResearch Leads: Zhenmei Shi, Akshay Goindani, Hong Liu Large language models are increasingly being used for a broad range of tasks, in…

Large language model

A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts, and are a foundational technology behind modern chatbots.[1] Biased or inaccurate training data can make an LLM's output less reliable.[2]

How to Make Small Language Models Outperform Large Language Models Using DSPy!

How a 3B Language Model Surpasses an 8B Counterpart with DSPy? “In an era where language models (LMs) are revolutionising countless tasks, their potential is only as powerful as we interpret …

Beyond Standard LLMs

Linear Attention Hybrids, Text Diffusion, Code World Models, and Small Recursive Transformers

Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via...

Despite their impressive capabilities, multilingual large language models (MLLMs) frequently exhibit inconsistent behaviour when the prompt's language changes. While such adaptation is generally...

Beyond Model Collapse: Scaling Up with Synthesized Data Requires...

Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is...

Statistical Foundations of LLM-based A/B Testing: A Surrogacy...

Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in the hope of experimenting faster and at lower cost. We...

MemGPT (MemGPT)

Large Language Models (LLMs)

The diffusion of large language models in published academic articles

Large language models (LLMs) are rapidly changing academic research, raising questions of who is adopting these tools and under what conditions. This article analyzes full texts of 7.3 million journal articles published from 2020–2025 by four major publishers (Elsevier, Frontiers, MDPI, and PLoS) to track the prevalence of LLM-associated language and identify social and institutional correlates of adoption. A corpus of 228 focal words exhibiting sharp post-2022 frequency increases consistent with LLM output was developed; articles were scored on their rate of focal word usage. By 2025, an estimated 57% of published articles exhibited evidence of LLM influence, up from 12% in 2023. Among articles exhibiting LLM-influenced text, there is substantial heterogeneity, ranging from subtle linguistic influence to articles mostly or entirely LLM-generated. Difference-in-differences models reveal that LLM-associated language varies markedly across regions, institutional ranks, publishers, disciplines, and journal tiers. Economic development and proximity to English as a primary language are key predictors of regional variation. Lower-ranked institutions exhibit higher rates than elite universities, young for-profit publishers show elevated rates vis-à-vis competitors, and academic fields differ widely in adoption. LLM adoption in academic writing is pervasive but socially stratified. As models grow more powerful and their use becomes further entrenched in academic research, understanding social dynamics of adoption will be essential for governing the evolving relationship between AI and academic knowledge production.

Introducing Mercury 2.5 – Inception

Mercury 2.5 is the most capable diffusion LLM on the market. It runs at 1,107 tokens/sec and offers a 40% increase in intelligence over Mercury 2, comparable to cost-optimized frontier models.