ICML WhisperKit: On-device Real-time ASR with Billion-Scale Transformers

\robotoslablightdots.tts Technical Report

Text-to-speech (TTS) systems have largely solved intelligibility on standard read-speech benchmarks. What users expect from a modern system is broader: expressive and controllable output, real-time synthesis, and coverage of neutral reading, emotional dialogue, paralinguistic events, singing, and general audio. Current systems pursue this goal along three roughly distinct technical routes, and each route has its own unresolved problem.

How OpenAI delivers low-latency voice AI at scale

How OpenAI rebuilt its WebRTC stack to power real-time Voice AI with low latency, global scale, and seamless conversational turn-taking.

Launching a free, open-source, on-device transcription app

TL;DR – Please try Moonshine Note Taker on your Mac! For years I’ve been telling people that AI wants to be local, that on-device models aren’t just a poor man’s alternative…

Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

Large language model (LLM) queries are predominantly processed by frontier models in centralized cloud infrastructure. Demand growth strains this paradigm faster than providers can scale. Two advances create an opportunity to rethink it: small, local LMs (<=20B active parameters) now achieve competitive performance to frontier models on many tasks, and local accelerators (e.g., Apple M4 Max) can host these models at interactive latencies. This raises the question: can local inference viably redistribute demand from centralized infrastructure? This requires measuring both whether local LMs can accurately answer real-world queries and whether they can do so efficiently on power-constrained devices (e.g., laptops). We propose intelligence per watt (IPW), task accuracy per unit of power, as a unified metric for the capability and efficiency of local inference across model-accelerator configurations. We evaluate 20+ state-of-the-art local LMs, 8 hardware accelerators (local and cloud), and 1M real-world single-turn chat and reasoning queries. For each query, we measure accuracy (local LM win rate against frontier models), energy, latency, and power. We find three key results. First, local LMs successfully answer 88.7% of these queries, with accuracy varying by domain. Second, longitudinal analysis from 2023-2025 shows IPW improved 5.3x, driven by both algorithmic and accelerator advances, with locally-serviceable query coverage rising from 23.2% to 71.3%. Third, local accelerators achieve at least 1.4x lower IPW than cloud accelerators running identical models, revealing significant headroom for local accelerator optimization. These findings demonstrate that local inference can meaningfully redistribute demand from centralized infrastructure for a substantial subset of queries, with IPW serving as the critical metric for tracking this transition.

Gemma 4 Technical Report

We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.

Introducing Pipette: A benchmarking suite for on-device intelligence — Blog

Meet Pipette, an open-source platform for reproducible on-device AI benchmarks across models, quantization, runtimes and hardware.

DeepL AI Platform: Translation, Voice & API

Explore our AI suite and get more done: Translate speech, text, and media, or integrate the DeepL API.

Announcing transcribe.cpp

Meet transcribe.cpp, a new open-source C/C++ speech-to-text inference library with portable, GPU-accelerated support for multiple STT models. Developed through Mozilla.ai's Builders in Residence program, it makes adding fast, local transcription to applications easier than ever.

Gabber - Build Realtime AI Apps that can see, hear, and speak

Low-latency inference for VLM, TTS, and STT with orchestration for making realtime apps.

Introducing Whisper – an open source voice note taking app! Record voice notes and transcribe them into lists, blogs, & more with AI. 100% free & open source. https://t.co/UZWGkUDJ6d

Introducing Whisper – an open source voice note taking app!Record voice notes and transcribe them into lists, blogs, & more with AI.100% free & open source. pic.twitter.com/UZWGkUDJ6d— Hassan (@nutlope) July 22, 2025

Realtime and audio | OpenAI API

Learn which realtime and audio guide to use for each speech application.

kwindla on Twitter / X

Local voice AI with a 235 billion parameter LLM. ✅- smart-turn v2- MLX Whisper (large-v3-turbo-q4)- Qwen3-235B-A22B-Instruct-2507-3bit-DWQ- KokoroAll models running local on an M4 mac. Max RAM usage ~110GB.Voice-to-voice latency is ~950ms. There are a couple of… pic.twitter.com/iYNQlb9JkI— kwindla (@kwindla) July 27, 2025