Announcing a new collaborative research paper on Confidential Inference, a set of tools to improve the security of our model weights and of our users' data
Darkbloom — Cost-Efficient Private AI Inference on Verified Macs
Encrypted inference on hardware-verified Apple Silicon. Comparable model performance, operator-blind privacy, and about 50% lower cost.
Darkbloom — Cost-Efficient Private AI Inference on Verified Macs
Encrypted inference on hardware-verified Apple Silicon. Comparable model performance, operator-blind privacy, and about 50% lower cost.
Darkbloom — Cost-Efficient Private AI Inference on Verified Macs
Encrypted inference on hardware-verified Apple Silicon. Comparable model performance, operator-blind privacy, and about 50% lower cost.
ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models
As large language models (LLMs) are used in sensitive fields, accurately verifying their computational provenance without disclosing their training datasets poses a significant challenge, particularly in regulated sectors such as healthcare, which have strict requirements for dataset use. Traditional approaches either incur substantial computational cost to fully verify the entire training process or leak unauthorized information to the verifier. Therefore, we introduce ZKPROV, a novel cryptographic framework allowing users to verify that the LLM's responses to their prompts are trained on datasets certified by the authorities that own them. Additionally, it ensures that the dataset's content is relevant to the users' queries without revealing sensitive information about the datasets or the model parameters. ZKPROV offers a unique balance between privacy and efficiency by binding training datasets, model parameters, and responses, while also attaching zero-knowledge proofs to the responses generated by the LLM to validate these claims. Our experimental results demonstrate sublinear scaling for generating and verifying these proofs, with end-to-end overhead under 3.3 seconds for models up to 8B parameters, presenting a practical solution for real-world applications. We also provide formal security guarantees, proving that our approach preserves dataset confidentiality while ensuring trustworthy dataset provenance.
Defeating Prompt Injections by Design
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an untrusted environment. However, LLM agents are vulnerable to prompt injection attacks when handling untrusted data. In this paper we propose CaMeL, a robust defense that creates a protective system layer around the LLM, securing it even when underlying models are susceptible to attacks. To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query; therefore, the untrusted data retrieved by the LLM can never impact the program flow. To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows by enforcing security policies when tools are called. We demonstrate effectiveness of CaMeL by solving $77\%$ of tasks with provable security (compared to $84\%$ with an undefended system) in AgentDojo. We release CaMeL at https://github.com/google-research/camel-prompt-injection.
Private Post-Training and Inference for Frontier Models
A technical deep dive of Silo, our local-like privacy stack for cloud-based training and inference of trillion-parameter models.
Towards Feasible, Private, Distributed LLM Inference
Exploring how the Secure Transformer Inference Protocol (STIP) protects inputs, outputs, and model weights with lightweight permutations enabling efficient, privacy-safe LLM inference at scale.
Gajesh on Twitter / X
TL;DRapple has turn on this switch for everyone to participate in decentralized inferenceppl can rent out their unused compute space and anyone can use this with privacy guarantees https://t.co/LTP4zyjsdt pic.twitter.com/8Dvo7XK8jJ— Gajesh (@gajesh) February 18, 2026
Membership Inference Attacks Against Machine Learning Models
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained. We focus on the basic membership inference attack: given a data record and black-box access to a model, determine if the record was in the model's training dataset. To perform membership inference against a target model, we make adversarial use of machine learning and train our own inference model to recognize differences in the target model's predictions on the inputs that it trained on versus the inputs that it did not train on. We empirically evaluate our inference techniques on classification models trained by commercial "machine learning as a service" providers such as Google and Amazon. Using realistic datasets and classification tasks, including a hospital discharge dataset whose membership is sensitive from the privacy perspective, we show that these models can be vulnerable to membership inference attacks. We then investigate the factors that influence this leakage and evaluate mitigation strategies.
Membership Inference Attacks Against Machine Learning Models
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained. We focus on the basic membership inference attack: given a data record and black-box access to a model, determine if the record was in the model's training dataset. To perform membership inference against a target model, we make adversarial use of machine learning and train our own inference model to recognize differences in the target model's predictions on the inputs that it trained on versus the inputs that it did not train on. We empirically evaluate our inference techniques on classification models trained by commercial "machine learning as a service" providers such as Google and Amazon. Using realistic datasets and classification tasks, including a hospital discharge dataset whose membership is sensitive from the privacy perspective, we show that these models can be vulnerable to membership inference attacks. We then investigate the factors that influence this leakage and evaluate mitigation strategies.
Piloting the world's first double-blind AI evaluations
Building trust in proprietary model benchmarks using cryptographically secure environments
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Machine learning algorithms, when applied to sensitive data, pose a distinct threat to privacy. A growing body of prior work demonstrates that models produced by these algorithms may leak specific private information in the training data to an attacker, either through the models' structure or their observable behavior. However, the underlying cause of this privacy risk is not well understood beyond a handful of anecdotal accounts that suggest overfitting and influence might play a role. This paper examines the effect that overfitting and influence have on the ability of an attacker to learn information about the training data from machine learning models, either through training set membership inference or attribute inference attacks. Using both formal and empirical analyses, we illustrate a clear relationship between these factors and the privacy risk that arises in several popular machine learning algorithms. We find that overfitting is sufficient to allow an attacker to perform membership inference and, when the target attribute meets certain conditions about its influence, attribute inference attacks. Interestingly, our formal analysis also shows that overfitting is not necessary for these attacks and begins to shed light on what other factors may be in play. Finally, we explore the connection between membership inference and attribute inference, showing that there are deep connections between the two that lead to effective new attacks.
Unlinkable Inference as a User Privacy Architecture
Unlinkable inference is a technique that provably sandboxes your AI requests from each other and from your identity. We discuss its building blocks, applications, and how it fits into the broader landscape of private personal intelligence.
Equivariance Encryption for Private LLM Inference — Davide Cifarelli
This article shows how we can leverage equivariant transformations to run LLM inference on encrypted data without losing model performance.
Security research on Private Cloud Compute - Apple Security Research
Private Cloud Compute (PCC) fulfills computationally intensive requests for Apple Intelligence while providing groundbreaking privacy and security protections — by bringing our industry-leading device security model into the cloud. To build public trust in our system, we’re making it possible for researchers to inspect and verify PCC’s security and privacy guarantees by releasing tools and resources including a comprehensive PCC Security Guide, the software binaries and source code of key PCC components, and — in a first for any Apple platform — a Virtual Research Environment, which allows anyone to install and test the PCC software on a Mac with Apple silicon.
