There’s software used across the country to predict future criminals. And it’s biased against blacks.
Predictive Policing Explained
Attempts to forecast crime with algorithmic techniques could reinforce existing racial biases in the criminal justice system.
Does Predictive Policing Lead to Biased Arrests? Results From a Randomized Controlled Trial
Racial bias in predictive policing algorithms has been the focus of a number of recent news articles, statements of concern by several national organizations (e.g., the ACLU and NAACP), and simulat...
Does Predictive Policing Lead to Biased Arrests? Results From a Randomized Controlled Trial
Racial bias in predictive policing algorithms has been the focus of a number of recent news articles, statements of concern by several national organizations (e.g., the ACLU and NAACP), and simulat...
Trial Data Reveals Racial Bias in Age Verification Software
New trial data reveals racial bias in age verification software used for social media restrictions. Discover how unreliable these systems really are now.
Certifying and Removing Disparate Impact
What does it mean for an algorithm to be biased? In U.S. law, unintentional bias is encoded via disparate impact, which occurs when a selection process has widely different outcomes for different groups, even as it appears to be neutral. This legal determination hinges on a definition of a protected class (ethnicity, gender) and an explicit description of the process.When computers are involved, determining disparate impact (and hence bias) is harder. It might not be possible to disclose the process. In addition, even if the process is open, it might be hard to elucidate in a legal setting how the algorithm makes its decisions. Instead of requiring access to the process, we propose making inferences based on the data it uses.We present four contributions. First, we link disparate impact to a measure of classification accuracy that while known, has received relatively little attention. Second, we propose a test for disparate impact based on how well the protected class can be predicted from the other attributes. Third, we describe methods by which data might be made unbiased. Finally, we present empirical evidence supporting the effectiveness of our test for disparate impact and our approach for both masking bias and preserving relevant information in the data. Interestingly, our approach resembles some actual selection practices that have recently received legal scrutiny.
How the LAPD and Palantir Use Data to Justify Racist Policing
In a new book, a sociologist who spent months embedded with the LAPD details how data-driven policing techwashes bias.
Predictive policing AI is on the rise − making it accountable to the public could curb its harmful effects
AI that anticipates where crimes are likely to occur and who might commit them has a troubling track record. Democratic accountability could shine a light on the technology and how it’s used.
Predictive policing AI is on the rise − making it accountable to the public could curb its harmful effects
AI that anticipates where crimes are likely to occur and who might commit them has a troubling track record. Democratic accountability could shine a light on the technology and how it’s used.
Artificial Intelligence, Predictive Policing, and Risk Assessment for Law Enforcement
There are widespread concerns about the use of artificial intelligence in law enforcement. Predictive policing and risk assessment are salient examples. Worries include the accuracy of forecasts that guide both activities, the prospect of bias, and an apparent lack of operational transparency. Nearly breathless media coverage of artificial intelligence helps shape the narrative. In this review, we address these issues by first unpacking depictions of artificial intelligence. Its use in predictive policing to forecast crimes in time and space is largely an exercise in spatial statistics that in principle can make policing more effective and more surgical. Its use in criminal justice risk assessment to forecast who will commit crimes is largely an exercise in adaptive, nonparametric regression. It can in principle allow law enforcement agencies to better provide for public safety with the least restrictive means necessary, which can mean far less use of incarceration. None of this is mysterious. Nevertheless, concerns about accuracy, fairness, and transparency are real, and there are tradeoffs between them for which there can be no technical fix. You can't have it all. Solutions will be found through political and legislative processes achieving an acceptable balance between competing priorities.
Artificial Intelligence, Predictive Policing, and Risk Assessment for Law Enforcement
There are widespread concerns about the use of artificial intelligence in law enforcement. Predictive policing and risk assessment are salient examples. Worries include the accuracy of forecasts that guide both activities, the prospect of bias, and an apparent lack of operational transparency. Nearly breathless media coverage of artificial intelligence helps shape the narrative. In this review, we address these issues by first unpacking depictions of artificial intelligence. Its use in predictive policing to forecast crimes in time and space is largely an exercise in spatial statistics that in principle can make policing more effective and more surgical. Its use in criminal justice risk assessment to forecast who will commit crimes is largely an exercise in adaptive, nonparametric regression. It can in principle allow law enforcement agencies to better provide for public safety with the least restrictive means necessary, which can mean far less use of incarceration. None of this is mysterious. Nevertheless, concerns about accuracy, fairness, and transparency are real, and there are tradeoffs between them for which there can be no technical fix. You can't have it all. Solutions will be found through political and legislative processes achieving an acceptable balance between competing priorities.
Algorithmic Bias · Open Encyclopedia of Cognitive Science
Algorithmic bias refers to prejudicial, discriminatory, unjust, inaccurate, or otherwise disparate performance or outcomes from algorithmic systems based on racial, gender, or other attributes of an individual or a group. The concept of algorithmic bias emerged at the intersection of computer science, artificial intelligence (AI) research, critical data studies, human–computer interaction, law, philosophy, and similar disciplines. Although problems and discrepancies at the model level denote the most commonly studied form of bias, the term algorithmic bias is also used as a shorthand to describe a multitude of problems and challenges at various steps of the AI pipeline from ideation, problem framing, training data curation and processing, model training and validation, and deployment as well as emergent issues that arise from interaction with the real world. Potential sources of bias, appropriate metrics to define, measure, and mitigate bias, and the utility and merit of technical approaches to bias mitigation are fiercely debated in the current AI landscape.
How AI can lead to false arrests and wrongful convictions
Danger arises when law enforcement believes that AI models are retrieving certainties rather than generating likelihoods.
How AI can lead to false arrests and wrongful convictions
Danger arises when law enforcement believes that AI models are retrieving certainties rather than generating likelihoods.
Black prisoners are assigned harsher living conditions in Ontario jails—thanks to AI ⋆ The Breach
Ontario jails are using a program that claims it can predict prisoners’ behaviour, disproportionately putting Black prisoners in higher-security facilities
What the latest research tells us about racial bias in policing
What we know, and importantly don't know, about the latest research on racial bias in policing.
Algorithmic Bias in Lending: Evidence from a Fintech Audit
Algorithmic lending has transformed the consumer credit landscape, with machine learning models commonly facilitating underwriting decisions. To comply with fair lending laws, these algorithms exclude legally protected characteristics, such as race and gender. Yet algorithmic underwriting can still inadvertently favor certain groups, prompting concerns about whether lending algorithms exhibit discriminatory behavior. Using proprietary loan-level data from a major U.S. fintech platform, we audit lending decisions across approximately 80,000 personal loans. We find that loans made to men and Black borrowers yielded lower profits than loans to other groups, suggesting that men and Black borrowers benefited from relatively favorable pricing. We trace these disparities to miscalibration in the platform's underwriting model, which overestimates risk for women and underestimates risk for Black borrowers. We then show that one could correct this miscalibration -- and the corresponding disparities -- by including race and gender in underwriting models, illustrating a tension between competing notions of fairness.
