User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies

Hundreds of millions of people now regularly interact with large language models via chatbots. Model developers are eager to acquire new sources of high-quality training data as they race to improve model capabilities and win market share. This paper analyzes the privacy policies of six U.S. frontier AI developers to understand how they use their users' chats to train models. Drawing primarily on the California Consumer Privacy Act, we develop a novel qualitative coding schema that we apply to each developer's relevant privacy policies to compare data collection and use practices across the six companies. We find that all six developers appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely. Developers may collect and train on personal information disclosed in chats, including sensitive information such as biometric and health data, as well as files uploaded by users. Four of the six companies we examined appear to include children's chat data for model training, as well as customer data from other products. On the whole, developers' privacy policies often lack essential information about their practices, highlighting the need for greater transparency and accountability. We address the implications of users' lack of consent for the use of their chat data for model training, data security issues arising from indefinite chat data retention, and training on children's chat data. We conclude by providing recommendations to policymakers and developers to address the data privacy challenges posed by LLM-powered chatbots.

User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies

Hundreds of millions of people now regularly interact with large language models via chatbots. Model developers are eager to acquire new sources of high-quality training data as they race to improve model capabilities and win market share. This paper analyzes the privacy policies of six U.S. frontier AI developers to understand how they use their users' chats to train models. Drawing primarily on the California Consumer Privacy Act, we develop a novel qualitative coding schema that we apply to each developer's relevant privacy policies to compare data collection and use practices across the six companies. We find that all six developers appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely. Developers may collect and train on personal information disclosed in chats, including sensitive information such as biometric and health data, as well as files uploaded by users. Four of the six companies we examined appear to include children's chat data for model training, as well as customer data from other products. On the whole, developers' privacy policies often lack essential information about their practices, highlighting the need for greater transparency and accountability. We address the implications of users' lack of consent for the use of their chat data for model training, data security issues arising from indefinite chat data retention, and training on children's chat data. We conclude by providing recommendations to policymakers and developers to address the data privacy challenges posed by LLM-powered chatbots.

Who Even Cares About Data Ownership Anyway? - What The Function!?

Taking a look at a concept that's been talked a lot over the last few years, especially in the context of social media

The Limits of Data

Policymakers want to make decisions based on clear data, but important factors are lost when we rely solely on data. A philosopher writes:

Dan Shipper 📧 on Twitter / X

The smallest technical decisions become humanity's biggest pivots:The same-origin policy—a well-intentioned browser security rule from the 1990s—accidentally created Facebook, Google, and every data monopoly since. It locks your data in silos—and you stayed where your stuff… pic.twitter.com/G87qsuPHxN— Dan Shipper 📧 (@danshipper) July 9, 2025

Living in Data: A Citizen's Guide to a Better Information Future (Paperback)

Jer Thorp’s analysis of the word “data” in 10,325 New York Times stories written between 1984 and 2018 shows a distinct trend: among the words most closely associated with “data,” we find not only its classic companions “information” and “digital,” but also a variety of new neighbors—from “scandal” and “misinformation” to “ethics,” “friends,” and “play.”To live in data in the twenty-first century

Data Minimisation: a Language-Based Approach (Long Version)

Data minimisation is a privacy-enhancing principle considered as one of the pillars of personal data regulations. This principle dictates that personal data collected should be no more than...

Technology is Not Values Neutral: Ending the Reign of Nihilistic Design - The Consilience Project

We use cookies on our website, please read our Privacy Policy for more information Okay, thanks

How One 1990s Browser Decision Created Big Tech’s Data Monopolies (And How We Might Finally Fix It) - Above the Law

From the take-back-control dept

A data minimization model for embedding privacy into software systems

Modern software systems (social networking, banking and shopping applications) are becoming increasingly dependent on our data. These systems need data to provide various economic and social benefits to users as well as businesses. However, the extensive use of personal data in systems poses a threat to user privacy. Therefore, privacy laws expect software systems to practice Data Minimization (DM), to minimize data in software systems. This has put software developers in a dilemma to minimize user data to provide user privacy and maximize user data for enhanced system functionality. Following the design science research approach, in this research we propose and evaluate a methodology that enables developers to make their decisions to minimize user data in software systems through understanding data. The methodology encourage developers to think of the ways they would use data in a system design focusing on the storage and sharing of data. Developers in the three experiments conducted to evaluate the methodology agreed that it enables them to think of the ways they use data in system designs and it helps them to make decisions to minimize using data in a system design. Developers also showed positive intention to use the proposed methodology within system development activities.

As tech companies race to build data centers, more communities are pushing back

Concerns about electricity bills and local impacts are fueling bipartisan opposition to the massive data centers that power the digital economy, from cloud services to AI chatbots. In Wisconsin, as in other states, the tussles are personal – and fraught.

Can “Conscious Data Contribution” Help Users to Exert “Data Leverage” Against Technology Companies?

Tech users currently have limited ability to act on concerns regarding the negative societal impacts of large tech companies. However, recent work suggests that users can exert leverage using their role in the generation of valuable data, for instance by withholding their data contributions to intelligent technologies. We propose and evaluate a new means to exert this type of leverage against tech companies: "conscious data contribution" (CDC).