User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies

Hundreds of millions of people now regularly interact with large language models via chatbots. Model developers are eager to acquire new sources of high-quality training data as they race to improve model capabilities and win market share. This paper analyzes the privacy policies of six U.S. frontier AI developers to understand how they use their users' chats to train models. Drawing primarily on the California Consumer Privacy Act, we develop a novel qualitative coding schema that we apply to each developer's relevant privacy policies to compare data collection and use practices across the six companies. We find that all six developers appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely. Developers may collect and train on personal information disclosed in chats, including sensitive information such as biometric and health data, as well as files uploaded by users. Four of the six companies we examined appear to include children's chat data for model training, as well as customer data from other products. On the whole, developers' privacy policies often lack essential information about their practices, highlighting the need for greater transparency and accountability. We address the implications of users' lack of consent for the use of their chat data for model training, data security issues arising from indefinite chat data retention, and training on children's chat data. We conclude by providing recommendations to policymakers and developers to address the data privacy challenges posed by LLM-powered chatbots.

User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies

Hundreds of millions of people now regularly interact with large language models via chatbots. Model developers are eager to acquire new sources of high-quality training data as they race to improve model capabilities and win market share. This paper analyzes the privacy policies of six U.S. frontier AI developers to understand how they use their users' chats to train models. Drawing primarily on the California Consumer Privacy Act, we develop a novel qualitative coding schema that we apply to each developer's relevant privacy policies to compare data collection and use practices across the six companies. We find that all six developers appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely. Developers may collect and train on personal information disclosed in chats, including sensitive information such as biometric and health data, as well as files uploaded by users. Four of the six companies we examined appear to include children's chat data for model training, as well as customer data from other products. On the whole, developers' privacy policies often lack essential information about their practices, highlighting the need for greater transparency and accountability. We address the implications of users' lack of consent for the use of their chat data for model training, data security issues arising from indefinite chat data retention, and training on children's chat data. We conclude by providing recommendations to policymakers and developers to address the data privacy challenges posed by LLM-powered chatbots.

Who Even Cares About Data Ownership Anyway? - What The Function!?

Taking a look at a concept that's been talked a lot over the last few years, especially in the context of social media

Who Even Cares About Data Ownership Anyway? - What The Function!?

Taking a look at a concept that's been talked a lot over the last few years, especially in the context of social media

Toward People’s Community Control of Technology: Race, Access, and Education

This field review explores how the benefits of access to computing for racialized and minoritized communities has become an accepted fact in policy and research, despite decades of evidence that technical fixes do not solve the kinds of complex social problems that disproportionately affect these communities.

Not OK, Google | TechCrunch

At its hardware launch event in San Francisco yesterday, Alphabet showed the sweeping breadth of its ambition to own consumers' personal data, as computing continues to accelerate away from static desktops and screens, coalescing into a cloud of connected devices with the potential to generate far more data -- and data of a far more intimate nature -- than ever before.

Can Software Make You Less Racist?

I don’t think we computer geeks appreciate how profoundly the rise of the smartphone, and Facebook, has changed the Internet audience. It’s something that really only happened in the last five years, as smartphones and data plans dropped radically in price and became accessible – and addictive – to huge

Rethinking how technology uses our personal data

This essay explores visually how keeping data and AI on‑device could empower not only users, but also companies.

The Limits of Data

Policymakers want to make decisions based on clear data, but important factors are lost when we rely solely on data. A philosopher writes:

Dan Shipper 📧 on Twitter / X

The smallest technical decisions become humanity's biggest pivots:The same-origin policy—a well-intentioned browser security rule from the 1990s—accidentally created Facebook, Google, and every data monopoly since. It locks your data in silos—and you stayed where your stuff… pic.twitter.com/G87qsuPHxN— Dan Shipper 📧 (@danshipper) July 9, 2025

Living in Data: A Citizen's Guide to a Better Information Future (Paperback)

Jer Thorp’s analysis of the word “data” in 10,325 New York Times stories written between 1984 and 2018 shows a distinct trend: among the words most closely associated with “data,” we find not only its classic companions “information” and “digital,” but also a variety of new neighbors—from “scandal” and “misinformation” to “ethics,” “friends,” and “play.”To live in data in the twenty-first century

Home - Data & Society

Data & Society believes that technology policy must be grounded in research, account for tech’s real-world impacts, and serve the public.

Data Minimisation: a Language-Based Approach (Long Version)

Data minimisation is a privacy-enhancing principle considered as one of the pillars of personal data regulations. This principle dictates that personal data collected should be no more than...