This article shows how we can leverage equivariant transformations to run LLM inference on encrypted data without losing model performance.
Towards Feasible, Private, Distributed LLM Inference
Exploring how the Secure Transformer Inference Protocol (STIP) protects inputs, outputs, and model weights with lightweight permutations enabling efficient, privacy-safe LLM inference at scale.
Private Post-Training and Inference for Frontier Models
A technical deep dive of Silo, our local-like privacy stack for cloud-based training and inference of trillion-parameter models.
zama-ai/concrete-ml
Concrete ML: Privacy Preserving ML framework using Fully Homomorphic Encryption (FHE), built on top of Concrete, with bindings to traditional ML frameworks.
ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models
As large language models (LLMs) are used in sensitive fields, accurately verifying their computational provenance without disclosing their training datasets poses a significant challenge, particularly in regulated sectors such as healthcare, which have strict requirements for dataset use. Traditional approaches either incur substantial computational cost to fully verify the entire training process or leak unauthorized information to the verifier. Therefore, we introduce ZKPROV, a novel cryptographic framework allowing users to verify that the LLM's responses to their prompts are trained on datasets certified by the authorities that own them. Additionally, it ensures that the dataset's content is relevant to the users' queries without revealing sensitive information about the datasets or the model parameters. ZKPROV offers a unique balance between privacy and efficiency by binding training datasets, model parameters, and responses, while also attaching zero-knowledge proofs to the responses generated by the LLM to validate these claims. Our experimental results demonstrate sublinear scaling for generating and verifying these proofs, with end-to-end overhead under 3.3 seconds for models up to 8B parameters, presenting a practical solution for real-world applications. We also provide formal security guarantees, proving that our approach preserves dataset confidentiality while ensuring trustworthy dataset provenance.
Confidential Inference via Trusted Virtual Machines
Announcing a new collaborative research paper on Confidential Inference, a set of tools to improve the security of our model weights and of our users' data
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
HEIR: Homomorphic Encryption Intermediate Representation
HEIR is a compiler toolchain for fully homomorphic encryption (FHE). We aim to be the industry-standard compiler for FHE. Application developers, compiler engineers, hardware designers, and cryptography researchers can build upon HEIR to accelerate the research and development of production-strength privacy-first software systems.
Gajesh on Twitter / X
TL;DRapple has turn on this switch for everyone to participate in decentralized inferenceppl can rent out their unused compute space and anyone can use this with privacy guarantees https://t.co/LTP4zyjsdt pic.twitter.com/8Dvo7XK8jJ— Gajesh (@gajesh) February 18, 2026
Machine Learning
Learn how Ente uses on-device machine learning to power private, end-to-end encrypted features like face recognition and semantic search—without a cloud.
VaultGemma: The world's most capable differentially private LLM
Amer Sinha, Software Engineer, and Ryan McKenna, Research Scientist, Google Research
Encrypted Spaces — Research preview
An architecture for collaborative applications where data is encrypted and operations are cryptographically verifiable.
