Modern software systems (social networking, banking and shopping applications) are becoming increasingly dependent on our data. These systems need data to provide various economic and social benefits to users as well as businesses. However, the extensive use of personal data in systems poses a threat to user privacy. Therefore, privacy laws expect software systems to practice Data Minimization (DM), to minimize data in software systems. This has put software developers in a dilemma to minimize user data to provide user privacy and maximize user data for enhanced system functionality. Following the design science research approach, in this research we propose and evaluate a methodology that enables developers to make their decisions to minimize user data in software systems through understanding data. The methodology encourage developers to think of the ways they would use data in a system design focusing on the storage and sharing of data. Developers in the three experiments conducted to evaluate the methodology agreed that it enables them to think of the ways they use data in system designs and it helps them to make decisions to minimize using data in a system design. Developers also showed positive intention to use the proposed methodology within system development activities.
Data Minimisation: a Language-Based Approach (Long Version)
Data minimisation is a privacy-enhancing principle considered as one of the pillars of personal data regulations. This principle dictates that personal data collected should be no more than...
User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies
Hundreds of millions of people now regularly interact with large language models via chatbots. Model developers are eager to acquire new sources of high-quality training data as they race to improve model capabilities and win market share. This paper analyzes the privacy policies of six U.S. frontier AI developers to understand how they use their users' chats to train models. Drawing primarily on the California Consumer Privacy Act, we develop a novel qualitative coding schema that we apply to each developer's relevant privacy policies to compare data collection and use practices across the six companies. We find that all six developers appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely. Developers may collect and train on personal information disclosed in chats, including sensitive information such as biometric and health data, as well as files uploaded by users. Four of the six companies we examined appear to include children's chat data for model training, as well as customer data from other products. On the whole, developers' privacy policies often lack essential information about their practices, highlighting the need for greater transparency and accountability. We address the implications of users' lack of consent for the use of their chat data for model training, data security issues arising from indefinite chat data retention, and training on children's chat data. We conclude by providing recommendations to policymakers and developers to address the data privacy challenges posed by LLM-powered chatbots.
User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies
Hundreds of millions of people now regularly interact with large language models via chatbots. Model developers are eager to acquire new sources of high-quality training data as they race to improve model capabilities and win market share. This paper analyzes the privacy policies of six U.S. frontier AI developers to understand how they use their users' chats to train models. Drawing primarily on the California Consumer Privacy Act, we develop a novel qualitative coding schema that we apply to each developer's relevant privacy policies to compare data collection and use practices across the six companies. We find that all six developers appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely. Developers may collect and train on personal information disclosed in chats, including sensitive information such as biometric and health data, as well as files uploaded by users. Four of the six companies we examined appear to include children's chat data for model training, as well as customer data from other products. On the whole, developers' privacy policies often lack essential information about their practices, highlighting the need for greater transparency and accountability. We address the implications of users' lack of consent for the use of their chat data for model training, data security issues arising from indefinite chat data retention, and training on children's chat data. We conclude by providing recommendations to policymakers and developers to address the data privacy challenges posed by LLM-powered chatbots.
Monitoring Data Minimisation
Data minimisation is a privacy enhancing principle, stating that personal data collected should be no more than necessary for the specific purpose consented by the user. Checking that a program...
Privacy Architectures: Reasoning About Data Minimisation and Integrity
Privacy by design will become a legal obligation in the European Community if the Data Protection Regulation eventually gets adopted. However, taking into account privacy requirements in the design of a system is a challenging task. We propose an approach based on the specification of privacy architectures and focus on a key aspect of privacy, data minimisation, and its tension with integrity requirements. We illustrate our formal framework through a smart metering case study.
Privacy and human behavior in the age of information
This Review summarizes and draws connections between diverse streams of empirical research on privacy behavior. We use three themes to connect insights from social and behavioral sciences: people's uncertainty about the consequences of privacy-related behaviors and their own preferences over those consequences; the context-dependence of people's concern, or lack thereof, about privacy; and the degree to which privacy concerns are malleable—manipulable by commercial and governmental interests. Organizing our discussion by these themes, we offer observations concerning the role of public policy in the protection of privacy in the information age.
Privacy by Design
In view of rapid and dramatic technological change, it is important to take the special requirements of privacy protection into account early on, because new technological systems often contain hidden dangers which are very difficult to overcome after the basic design has been worked out. So it makes all the more sense to identify and examine possible data protection problems when designing new technology and to incorporate privacy protection into the overall design, instead of having to come up with laborious and time-consuming “patches” later on. This approach is known as “Privacy by Design” (PbD).
Privacy by design: a formal framework for the analysis of architectural choices
The privacy by design approach has already been applied in different areas. We believe that the next challenge in this area today is to go beyond individual cases and to provide methodologies to explore the design space in a systematic way. As a first step in this direction, we focus in this paper on the data minimization principle and consider different options using decentralized architectures in which actors do not necessarily trust each other. We propose a framework to express the parameters to be taken into account (the service to be performed, the actors involved, their respective requirements, etc.) and an inference system to derive properties such as the possibility for an actor to detect potential errors (or frauds) in the computation of a variable. This inference system can be used in the design phase to check if an architecture meets the requirements of the parties or to point out conflicting requirements.
Research | Tiles Book
Progressive experiments exploring decentralized software technologies for privacy adoption at scale.
Data Minimisation in Communication Protocols: A Formal Analysis...
With the growing amount of personal information exchanged over the Internet, privacy is becoming more and more a concern for users. One of the key principles in protecting privacy is data...
Local-first software: You own your data, in spite of the cloud
A new generation of collaborative software that allows users to retain ownership of their data.
Local-first software: You own your data, in spite of the cloud
A new generation of collaborative software that allows users to retain ownership of their data.
Measuring the Privacy Experience
How do you know that a product respects your privacy – other than by wading through the fine print? We’ve created a framework to measure the way people actually experience privacy in tech products.
Nonrivalry and the Economics of Data
(September 2020) - Data is nonrival: a person's location history, medical records, and driving data can be used by many firms simultaneously. Nonrivalry leads to increasing returns. As a result, there may be social gains to data being used broadly across firms, even in the presence of privacy considerations. Fearing creative destruction, firms may choose to hoard their data, leading to the inefficient use of nonrival data. Giving data property rights to consumers can generate allocations that are close to optimal. Consumers balance their concerns for privacy against the economic gains that come from selling data broadly.
Data Leverage: A Framework for Empowering the Public in its Relationship with Technology Companies
Many powerful computing technologies rely on implicit and explicit data contributions from the public. In this paper, we synthesize emerging research that seeks to better understand and help people action this data leverage. Drawing on prior work in areas including machine learning, human-computer interaction, and fairness and accountability in computing, we present a framework for understanding data leverage that highlights new opportunities to change technology company behavior related to privacy, economic inequality, content moderation and other areas of societal concern.