How we cryptographically guarantee that we are serving specific, untampered model weights that clients can verify on each request.
Backend infrastructure - Tinfoil Documentation
This page provides a description of the different components that make up our backend infrastructure. It also describes how Tinfoil guarantees code auditability and data confidentiality using these components.
Introducing Tinfoil Containers
Tinfoil Containers lets you deploy your application backend, your training pipeline, or your proprietary model on Tinfoil and build end-to-end verifiably private AI services.
Building a Browser-Native Verification Stack for Tinfoil
Learn how we built a browser-based confidentiality and integrity verifier with implementations of browser-native Sigstore and TUF libraries.
Tinfoil - Private AI
AI that keeps your data private at all times. Fast, powerful, and verifiable, thanks to secure hardware enclaves.
Confidential Inference via Trusted Virtual Machines
Announcing a new collaborative research paper on Confidential Inference, a set of tools to improve the security of our model weights and of our users' data
Tinfoil Private Chat
Private AI chat application supporting open source models through Tinfoil
Automated Verification of Proofs in the Universal Composability Framework with Markov Decision Processes
Designing cryptographic protocols and proving these rigorously secure is an arduous and challenging task. Among the methods commonly used to prove security of cryptographic protocols, formalizing it in Canneti's Universal Composability (UC) Framework offers several benefits: (1) Modular design, (2) demonstrating that security remains under arbitrary composition and concurrent execution, (3) the security against any computationally polynomially bound adversary. However, working within the UC Framework can be cumbersome, requires a long time commitment by the prover, and it is prone to errors. While utilization of proof assistants in Cryptography and IT Security is a prominent research area, proof assistants for UC are still in their infancy. Here we show our ongoing work to utilize model checking for verification of proofs in the UC Framework, which to the best of our knowledge is the first attempt to do so. In this work we (1) formally create a Markov Decision Process (MDP) encoding a given proof in the UC Framework, (2) define and proof notions of soundness and completeness for the constructed MDP, (3) implement a proof of concept and (4) demonstrate practical feasibility through experimental evaluation. In summary, in this work we lay out the formal foundations for model checking UC proofs and create a tool that can not only be used for proof verification but also as an assistant for developing proofs in the UC Framework.
CIDs: What You Need to Know and Why, Part 1 - Nick's Blog
A deep dive into Content Identifiers, the self-describing cryptographic fingerprints that form the foundation of ATProtocol’s data model.
ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models
As large language models (LLMs) are used in sensitive fields, accurately verifying their computational provenance without disclosing their training datasets poses a significant challenge, particularly in regulated sectors such as healthcare, which have strict requirements for dataset use. Traditional approaches either incur substantial computational cost to fully verify the entire training process or leak unauthorized information to the verifier. Therefore, we introduce ZKPROV, a novel cryptographic framework allowing users to verify that the LLM's responses to their prompts are trained on datasets certified by the authorities that own them. Additionally, it ensures that the dataset's content is relevant to the users' queries without revealing sensitive information about the datasets or the model parameters. ZKPROV offers a unique balance between privacy and efficiency by binding training datasets, model parameters, and responses, while also attaching zero-knowledge proofs to the responses generated by the LLM to validate these claims. Our experimental results demonstrate sublinear scaling for generating and verifying these proofs, with end-to-end overhead under 3.3 seconds for models up to 8B parameters, presenting a practical solution for real-world applications. We also provide formal security guarantees, proving that our approach preserves dataset confidentiality while ensuring trustworthy dataset provenance.
Defeating Prompt Injections by Design
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an untrusted environment. However, LLM agents are vulnerable to prompt injection attacks when handling untrusted data. In this paper we propose CaMeL, a robust defense that creates a protective system layer around the LLM, securing it even when underlying models are susceptible to attacks. To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query; therefore, the untrusted data retrieved by the LLM can never impact the program flow. To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows by enforcing security policies when tools are called. We demonstrate effectiveness of CaMeL by solving $77\%$ of tasks with provable security (compared to $84\%$ with an undefended system) in AgentDojo. We release CaMeL at https://github.com/google-research/camel-prompt-injection.
Public Persuasion with Endogenous Fact-Checking
We study public persuasion when a sender communicates with a large audience that can fact-check at heterogeneous costs. The sender commits to a public information policy before the state is realized, but any verifiable claim she makes after observing the state must be truthful (an ex-post implementability constraint). Receivers observe the public message and then decide whether to verify; this selective verification feeds back into the sender's objective and turns the design problem into a constrained version of Bayesian persuasion. Our main result is a reverse comparative static: when fact-checking becomes cheaper in the population, the sender optimally supplies a strictly less informative public signal. Intuitively, cheaper verification makes bold claims invite scrutiny, so the sender coarsens information to dampen the incentive to verify. We also endogenize two ex-post instruments - continuous falsification and fixed-cost repression - and characterize threshold substitutions from persuasion to manipulation and, ultimately, to repression as monitoring improves. The framework provides testable predictions for how transparency, manipulation, and repression co-move with changes in verification technology.
Open Weight AI Models Explained for Everyone
Introducing attested.network: Proof of Payment for ATProtocol - Nick's Blog
attested.network is an open spec for decentralized proof of payments on ATProtocol, built on what we learned making atprotofans.com. It formalizes the three-party attestation model and opens it up for any app to implement.
Trust Infrastructure on ATproto
I agree with Nick that you have noted a big hard problem, with Zooko’s triangle type trade-offs. (For a semi-related example that is “easier” but still crazy difficult, see our “PeerFlow: Secure Load Balancing in Tor” https://doi.org/10.1515/popets-2017-0017 ) As noted or implied already in this thread: in the end, if someone can cheaply spin up indefinite unlinked identities and build good reputations for them, and if trust is based on such reputations, it is hard (impossible?) to avoid them s...
Model Context Protocol has prompt injection security problems
As more people start hacking around with implementations of MCP (the Model Context Protocol, a new standard for making tools available to LLM-powered systems) the security implications of tools built …
