LLM inference in C/C++
llama.cpp/docs/build.md at master · ggml-org/llama.cpp
LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.
llama.cpp/tools/server at master · ggml-org/llama.cpp
LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.
llama.cpp with ROCm
WarningThis is a technical guide and assumes a certain level of technical knowledge. If there are confusing parts or you run into issues, I recommend using a strong LLM with research/grounding and reasoning abilities (eg Claude Sonnet 4) to assist.…
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Compiling knowledge, not retrieving it: a hands-on deep dive into llm-wiki-compiler
The thesis of this piece is simple and uncomfortable: the problem of making an LLM “remember” what you’ve read isn’t solved with more…
Can LLMs Be Computers? | Percepta
We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.
Can LLMs Be Computers? | Percepta
We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.
ggml
AI inference at the edge. ggml has 22 repositories available. Follow their code on GitHub.
LM Studio 0.3.10: 🔮 Speculative Decoding
Inference speed up with Speculative Decoding for `llama.cpp` and `MLX`
Defeating Nondeterminism in LLM Inference
Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models. For example, you might observe that asking ChatGPT the same question multiple times provides different results. This by itself is not surprising, since getting a result from a language model involves “sampling”, a process that converts the language model’s output into a probability distribution and probabilistically selects a token. What might be more surprising is that even when we adjust the temperature down to 0This means that the LLM always chooses the highest probability token, which is called greedy sampling. (thus making the sampling theoretically deterministic), LLM APIs are still not deterministic in practice (see past discussions here, here, or here). Even when running inference on your own hardware with an OSS inference library like vLLM or SGLang, sampling still isn’t deterministic (see here or here).
chad/whichlang
What programming language do LLMs default to when you don't tell them? A small benchmark.
GGML and llama.cpp join HF to ensure the long-term progress of Local AI
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
I just came across this paper arxiv.org/abs/2509.22908 which coins the term "vericoding" for using LLMs to generate formally verified code, and presents benchmark results for several languages
A benchmark for vericoding: formally verified program synthesis
arxiv.org
Martin Kleppmann
New blog post! ✨ I argue that AI will make formal verification go mainstream. martin.kleppmann.com/2025/12/08/ai-formal-verifica… Three reasons for this: 1. LLMs are getting increasingly good at writing proofs using proof assistants. This will make formal verification vastly cheaper than it's been to date.