Tomasz Limisiewicz on Twitter / X

Targeted Multilingual Adaptation for Low-resource Language Families

The "massively-multilingual" training of multilingual models is known to limit their utility in any one language, and they perform particularly poorly on low-resource languages. However, there is evid

INIYSA on Twitter / X

Apple's on-device SLM, while not as strong in simple multilingual tasks as Google's Gemma 3 4B (3.3GB, 5GB in memory), seems to outperform Qwen3 4B or Phi 4 mini reasoning. It's very very impressive, especially considering its extremely reduced size of around ~1.4GB— INIYSA (@lafaiel) June 10, 2025

DeepL Translator | World's Most Accurate Translator

Translate texts & full document files instantly. Accurate translations for individuals and Teams. Millions translate with DeepL every day.

Better Than Free

[Translations: Belarusian, Chinese, Estonian, French, German, Italian, Japanese, Polish, Portuguese, Russian, Spanish, Turkish] The internet is a copy machine. At its most foundational level, it copies every action, every character, every thought we make while we ride upon it. In order to … Continue reading →

A recent experience with ChatGPT 5.5 Pro

We are all having to keep revising upwards our assessments of the mathematical capabilities of large language models. I have just made a fairly large revision as a result of ChatGPT 5.5 Pro, to whi…

MemGPT (MemGPT)

Large Language Models (LLMs)

Julie Kallini ✈️ ICML✨ on Twitter / X

Fast Byte Latent Transformer is accepted to ICML 2026! ⚡🥪Byte-level LMs promise to free us from subword tokenizers, but decoding one byte at a time is super slow.We make BLT generation more efficient with BLT-D: text diffusion for parallel byte decoding. 1/ pic.twitter.com/ZIvUgavXvt— Julie Kallini ✈️ ICML✨ (@JulieKallini) May 11, 2026

Wiki-40B: Multilingual Language Model Dataset

We propose a new multilingual language model benchmark that is composed of 40+ languages spanning several scripts and linguistic families. With around 40 billion characters, we hope this new resource will accelerate the research of multilingual modeling. We train monolingual causal language models using a state-of-the-art model (Transformer-XL) establishing baselines for many languages. We also introduce the task of multilingual causal language modeling where we train our model on the combined text of 40+ languages from Wikipedia with different vocabulary sizes and evaluate on the languages individually. We released the cleaned-up text of 40+ Wikipedia language editions, the corresponding trained monolingual language models, and several multilingual language models with different fixed vocabulary sizes.

ChatGPT Translate | Fast, Natural, 40+ Languages

ChatGPT translates across 40+ languages with accuracy, tone, and cultural nuance. Translate text, voice, or photos for everyday use, travel, school, and work — and learn grammar or phrasing as you go.

How to Make Small Language Models Outperform Large Language Models Using DSPy!

How a 3B Language Model Surpasses an 8B Counterpart with DSPy? “In an era where language models (LMs) are revolutionising countless tasks, their potential is only as powerful as we interpret …

RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization

This study addresses the challenge of extending Large Language Models (LLMs) to non-English languages, specifically those using non-Roman scripts. We propose an approach that utilizes the romanized form of text as an interface for LLMs, hypothesizing that its frequent informal use and shared tokens with English enhance cross-lingual alignment. Our approach involve the continual pretraining of a English LLM like Llama 2 on romanized text of non-English, non-Roman script languages, followed by instruction tuning on romanized data. The results indicate that romanized text not only reduces token fertility by 2x-4x but also matches if not outperforms native script representation across various NLU, NLG and MT tasks. Moreover, the embeddings computed on romanized text exhibit closer alignment with their English translations than those from the native script. Our approach presents a promising direction for leveraging the power of English LLMs in languages traditionally underrepresented in NLP research.

What Language is This? Ask Your Tokenizer

Language Identification (LID) is an important component of many multilingual natural language processing pipelines, where it facilitates corpus curation, training data analysis, and cross-lingual evaluation of large language models. Despite near-perfect performance on high-resource languages, existing systems remain brittle in low-resource and closely related language settings. We introduce UniLID, a simple and efficient LID method based on the UnigramLM tokenization algorithm, leveraging its probabilistic framing, parameter estimation technique and inference strategy. In short, to predict a string's language label, we simply ask: under which language's unigram distribution is this string most likely? Our formulation is data- and compute-efficient, supports incremental addition of new languages without retraining existing models, and can naturally be integrated into existing language model tokenization pipelines. Empirical evaluations against widely used baselines, including fastText, GlotLID and CLD3, show that UniLID achieves competitive performance on standard benchmarks, substantially improves sample efficiency in low-resource settings -- reaching ~70% accuracy with as few as five labeled samples per language -- and delivers large gains on fine-grained dialect identification.

Post-Training Language Models for Crosslingual Consistency

Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model's response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback-Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages.

DSPy on Twitter / X

DSPy is the highest-bandwidth language to talk to computers in a just-precise-enough way.First 36 lines below:1. Take arbitrarily long content: `chunks`.2. Build a global Table of Contents.3. Distribute chunks into sections to be written.4. Recursively write each section. pic.twitter.com/7NN61isFLs— DSPy (@DSPyOSS) May 13, 2025

Pricing · Ollama

Get up and running with large language models.