Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1× and 2× cards.
club-3090/docs/MULTI_CARD.md at master · noonghunna/club-3090
Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B c...
noname on Twitter / X
Upto 1100 tps on RTX 3090x2 for Diffusion Gemma 4 26B.Unleash this mini monster on your gpus now!If you are running nvidia gpus locally, come grab the recipe at club-3090. https://t.co/qKuFcgu1llP.S. a ⭐️ on Github is much appreciated.@googlegemma @vllm_project— noname (@malikwas1f) June 11, 2026
Joel - coffee/acc on Twitter / X
OKAY - it seemed like DFlash would be the clear winner.But it appears there have been some improvements with MTP.With MTP + split-mode = tensor, Qwen3.6-27B gets over 120 tokens/second on dual RTX 3090s (note I am running PCIE x16 on both, I don't have an NVLink bridge).… https://t.co/AQmqTJrof7 pic.twitter.com/RQSPVsNwMO— Joel - coffee/acc (@JoelDeTeves) June 29, 2026
qualcomm/nexa-sdk
Run frontier LLMs and VLMs with day-0 model support across GPU, NPU, and CPU, with comprehensive runtime coverage for PC (Python/C++), mobile (Android & iOS), and Linux/IoT (Arm64 & x86 Docker). Supporting OpenAI GPT-OSS, IBM Granite-4, Qwen-3-VL, Gemma-3n, Ministral-3, and more.
OpenRouter
The unified interface for LLMs. Find the best models & prices for your prompts
crawshaw - 2026-02-08
I wrote up my experiences programming with LLMs a bit over a year ago, and updated it for the world of agents eight months ago. A lot has changed since then, so here is an update.
Xuan-Son Nguyen on Twitter / X
Firefox is open-source on Github, and they experimented with @ggml_org llama.cpp in WASM 👀Wondering what they are cooking 🧑🍳 pic.twitter.com/mMtVZo0gc2— Xuan-Son Nguyen (@ngxson) May 13, 2025
Benjamin Marie (@bnjmnmarie)
A new alternative to Ollama: You can now run models directly through Unsloth (with Docker): https://docs.unsloth.ai/models/how-to-run-llms-with-docker It supports the same models as llama.cpp, which I guess means it runs on llama.cpp… But this way you don’t need to set up anything, if you already have Docker installed.
BerriAI/litellm
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
Georgi Gerganov on Twitter / X
Introducing LlamaBarn — a tiny macOS menu bar app for running local LLMsOpen source, built on llama.cpp pic.twitter.com/F1Z3DVl9Kg— Georgi Gerganov (@ggerganov) January 29, 2026
at://userandagents.org/network.cosmik.card/Kffy9lJlvKgl-KCjGSTHymU4
[Tool Release] Finetune & Quantize 1–3B LLMs on 8GB RAM using LoFT CLI (TinyLlama + QLoRA + llama.cpp)
23 votes, 16 comments. Hey folks — I’ve been working on a CLI tool called LoFT (Low-RAM Finetuning Toolkit), and I finally have a working release. 🔧…