Introduction - How to Write an Inference Engine

videlalvaro/ane-book

Production LLM inference on the Apple Neural Engine — a practitioner's guide, complete with converters, Swift runtimes, and validated model manifests

Muse Glimmer: Meta’s 30B Model Built for Efficient Inference

Inside Meta’s 30B local reasoning model and its tiny KV cache

Introducing Muse Spark 1.3

Introducing Muse Spark 1.3, with max reasoning for challenging reasoning and agentic tasks and improved real-world usability.

Muse Spark 1.3 (max) - Intelligence, Performance & Price Analysis | Artificial Analysis

Analysis of Meta's Muse Spark 1.3 (max) and comparison to other AI models across key metrics including quality, price, performance (tokens per second & time to first token), context window & more.

antirez/ds4

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

Muse Spark 1.3 (xhigh) - Intelligence, Performance & Price Analysis | Artificial Analysis

Analysis of Meta's Muse Spark 1.3 (xhigh) and comparison to other AI models across key metrics including quality, price, performance (tokens per second & time to first token), context window & more.

Making Apple Neural Engine work in a custom inference stack

Apple Neural Engine always looked appealing on paper, but using it inside a custom runtime was harder. In 1.20260410.1, we made ANE practical for 8-bit S models by using CoreML only as an accelerator.

reinterpretcat/qwen3-rs

An educational Rust project for exporting and running inference on Qwen3 LLM family

Overview - GroqDocs

Fast LLM inference, OpenAI-compatible. Simple to integrate, easy to scale. Start building in minutes.

Open Models Inference for Coding · Umans AI

Hosted Kimi K3, GLM 5.2, and DeepSeek V4 Flash. Pay per token, on infrastructure we own.

Claude support for Apple's Foundation Models framework | Claude

A new Swift package connects Apple's Foundation Models framework to Claude. Hand off complex reasoning from on-device models with typed Swift outputs.

Claude Opus 4.8: Capabilities and Reactions

You need a lot of data points to understand a new model, and what you have.

Benjamin Marie on Twitter / X

Quantization and Qwen3-VL✔️ AutoRound W4A16 (INT4)✔️ AutoRound NVFP4 (llm compressor format)❌ support by vLLM (tried stable and dev releases)VLMs are now very easy to quantize. And then we can’t use the models with fast inference frameworks like vLLM and SGLang.It…— Benjamin Marie (@bnjmn_marie) November 5, 2025

uzu - Mirai Docs

Inference engine

ZML - Model to Metal

ZML is a production inference stack, purpose-built to decouple AI workloads from proprietary hardware.