Thousands of audiovisual images documented the insurrectionists who stormed the United States Capitol on January 6, 2021. Authorities subsequently collected those images and charged some for their criminal conduct. Given the overwhelming audiovisual evidence implicating the insurrectionists, it should be impossible to assert a plausible defense claiming that those unmistakably depicted in the images were not present. Right? Wrong. As the defense in the federal criminal trial of January 6th insurrectionist leader Guy Reffitt illustrated, the emergence of “deepfakes” has changed the landscape of plausible defenses to crimes. Reffitt led the attack on the Capital. Videos and other visual images showed him at the head of the crowd advancing on the Capitol’s West Terrace. He was arrested and charged with multiple crimes. And although the evidence, including audiovisual images, against Reffitt, was clear and overwhelming, his lawyer undermined it, arguing to the jury that the evidence against Reffitt was a “deepfake” – an audiovisual recording created using Artificial Intelligence technology that allows anyone with a smartphone to believably map one person’s movements and words onto the image of another person. Unfortunately, the law does not provide a clear response to Reffitt’s lawyer’s reliance on deepfakes as a defense. <br><br>But this much is clear—the “deepfake defense” is a new challenge to our legal system’s adversarial process and truth-seeking function. Because the norms of professional ethics require lawyers to advocate zealously, deepfakes invite lawyers to raise objections and arguments to evidence to exploit juror bias and skepticism about what is real. Thus, lawyers may plant the seeds of doubt in jurors’ minds to question the authenticity of all digital audio and visual images, even those counsel knows to be genuine.<br><br>Currently, no rule of procedure, ethics, or legal precedent directly addresses the presentation of the “deepfake defense” in court. The existing standards provide scant guidance because they were developed before the advent of deepfake technology. As a result, they do not solve the concern of how to deter lawyers from exploiting it. Although in the last several years, legal scholarship and the popular news media have addressed certain facets of deepfakes, there has been no in-depth commentary on the “deepfake defense.” This article is the first to explore the deepfake defense, locating it within the historical and current framework of lawyers’ efforts to fabricate evidence and the laws and the practice norms that exist to curb that conduct. It proposes a reconsideration of the ethical rules governing candor, fairness, and the limits of zealous advocacy and urges a re-examination of the court’s role in sanctioning such conduct. Thus, this article offers novel proposals to guide the way forward for lawyers and courts as they traverse this new technological landscape.
The Deepfake Defense—Exploring the Limits of the Law and Ethical Norms in Protecting Legal Proceedings from Lying Lawyers
Thousands of audiovisual images documented the insurrectionists who stormed the United States Capitol on January 6, 2021. Authorities subsequently collected tho
Deepfake detection by human crowds, machines, and machine-informed crowds
Significance The recent emergence of deepfake videos raises theoretical and practical questions. Are humans or the leading machine learning model more capable of detecting algorithmic visual manipulations of videos? How should content moderation systems be designed to detect and flag video-based misinformation? We present data showing that ordinary humans perform in the range of the leading machine learning model on a large set of minimal context videos. While we find that a system integrating human and model predictions is more accurate than either humans or the model alone, we show inaccurate model predictions often lead humans to incorrectly update their responses. Finally, we demonstrate that specialized face processing and the ability to consider context may specially equip humans for deepfake detection. , The recent emergence of machine-manipulated media raises an important societal question: How can we know whether a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers with the leading computer vision deepfake detection model and find them similarly accurate, while making different kinds of mistakes. Together, participants with access to the model’s prediction are more accurate than either alone, but inaccurate model predictions often decrease participants’ accuracy. To probe the relative strengths and weaknesses of humans and machines as detectors of deepfakes, we examine human and machine performance across video-level features, and we evaluate the impact of preregistered randomized interventions on deepfake detection. We find that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.
Deepfake detection by human crowds, machines, and machine-informed crowds
Significance The recent emergence of deepfake videos raises theoretical and practical questions. Are humans or the leading machine learning model more capable of detecting algorithmic visual manipulations of videos? How should content moderation systems be designed to detect and flag video-based misinformation? We present data showing that ordinary humans perform in the range of the leading machine learning model on a large set of minimal context videos. While we find that a system integrating human and model predictions is more accurate than either humans or the model alone, we show inaccurate model predictions often lead humans to incorrectly update their responses. Finally, we demonstrate that specialized face processing and the ability to consider context may specially equip humans for deepfake detection. , The recent emergence of machine-manipulated media raises an important societal question: How can we know whether a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers with the leading computer vision deepfake detection model and find them similarly accurate, while making different kinds of mistakes. Together, participants with access to the model’s prediction are more accurate than either alone, but inaccurate model predictions often decrease participants’ accuracy. To probe the relative strengths and weaknesses of humans and machines as detectors of deepfakes, we examine human and machine performance across video-level features, and we evaluate the impact of preregistered randomized interventions on deepfake detection. We find that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.
Deep Fakes: A Looming Challenge for Privacy
<p>Harmful lies are nothing new. But the ability to distort reality has taken an exponential leap forward with “deep fake” technology. This capability makes it possible to create audio and video of real people saying and doing things they never said or did. Machine learning techniques are escalating the technology’s sophistication, making deep fakes ever more realistic and increasingly resistant to detection. Deep-fake technology has characteristics that enable rapid and widespread diffusion, putting it into the hands of both sophisticated and unsophisticated actors.</p><p>While deep-fake technology will bring certain benefits, it also will introduce many harms. The marketplace of ideas already suffers from truth decay as our networked information environment interacts in toxic ways with our cognitive biases. Deep fakes will exacerbate this problem significantly. Individuals and businesses will face novel forms of exploitation, intimidation, and personal sabotage. The risks to our democracy and to national security are profound as well.</p><p>Our aim is to provide the first in-depth assessment of the causes and consequences of this disruptive technological change, and to explore the existing and potential tools for responding to it. We survey a broad array of responses, including: the role of technological solutions; criminal penalties, civil liability, and regulatory action; military and covert-action responses; economic sanctions; and market developments. We cover the waterfront from immunities to immutable authentication trails, offering recommendations to improve law and policy and anticipating the pitfalls embedded in various solutions.</p>
Deep Fakes: A Looming Challenge for Privacy
<p>Harmful lies are nothing new. But the ability to distort reality has taken an exponential leap forward with “deep fake” technology. This capability makes it possible to create audio and video of real people saying and doing things they never said or did. Machine learning techniques are escalating the technology’s sophistication, making deep fakes ever more realistic and increasingly resistant to detection. Deep-fake technology has characteristics that enable rapid and widespread diffusion, putting it into the hands of both sophisticated and unsophisticated actors.</p><p>While deep-fake technology will bring certain benefits, it also will introduce many harms. The marketplace of ideas already suffers from truth decay as our networked information environment interacts in toxic ways with our cognitive biases. Deep fakes will exacerbate this problem significantly. Individuals and businesses will face novel forms of exploitation, intimidation, and personal sabotage. The risks to our democracy and to national security are profound as well.</p><p>Our aim is to provide the first in-depth assessment of the causes and consequences of this disruptive technological change, and to explore the existing and potential tools for responding to it. We survey a broad array of responses, including: the role of technological solutions; criminal penalties, civil liability, and regulatory action; military and covert-action responses; economic sanctions; and market developments. We cover the waterfront from immunities to immutable authentication trails, offering recommendations to improve law and policy and anticipating the pitfalls embedded in various solutions.</p>
Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security — California Law Review
Harmful lies are nothing new. But the ability to distort reality has taken an exponential leap forward with “deep fake” technology. This capability makes it possible to create audio and video of real people saying and doing things they never said or did. Machine learning techniques are escalating th
An Indian politician says scandalous audio clips are AI deepfakes. We had them tested
Deepfake experts noted that AI can be used as a cover by politicians when embarrassing audio and video clips of them emerge.
People are trying to claim real videos are deepfakes. The courts are not amused
The unleashing of powerful, generative AI on the public is raising concerns that as the technology becomes more prevalent, it will become easier to claim that anything is fake.
Copy of Digital Masquerade
The Digital Masquerade: Unmasking AI’s Phantom Journalists By Tony Eastin and Sandeep Abraham, CAMS
New evidence, new challenges: ICC judges’ perspectives on user-generated evidence and judging in an age of artificial intelligence
Evidence recorded on personal digital devices, or “user-generated evidence” (UGE), has profoundly shaped our ways of knowing about international crimes. UGE can be expected to play an important role in future cases before the International Criminal Court (ICC), yet few trials to date have relied extensively on UGE.. This research provides important insights into how ICC judges define UGE and perceive its strengths and weaknesses, and on the readiness of the Court to adapt to judging in an age of Artificial Intelligence. Using grounded theory to analyse interviews with ICC judges, we identified several key themes, including concerns about the perceived importance and potential bias of evidence sources; the practical challenges of employing UGE; the burden placed on the parties to ensure the reliability of the evidence, to rigorously challenge the opposing party’s evidence, and the importance of preparing legal professionals to address the risks associated with misinformation and disinformation.
The Age of Realtime Deepfake Fraud Is Here
Fraudsters are able to change their race, facial hair, voice, and more during live video calls with very little effort. Scammers are already fooling the elderly and verification systems.
Going beyond the “common suspects”: to be presumed innocent in the era of algorithms, big data and artificial intelligence
This article explores the trend of increasing automation in law enforcement and criminal justice settings through three use cases: predictive policing, machine evidence and recidivism algorithms. The focus lies on artificial-intelligence-driven tools and technologies employed, whether at pre-investigation stages or within criminal proceedings, in order to decode human behaviour and facilitate decision-making as to whom to investigate, arrest, prosecute, and eventually punish. In this context, this article first underlines the existence of a persistent dilemma between the goal of increasing the operational efficiency of police and judicial authorities and that of safeguarding fundamental rights of the affected individuals. Subsequently, it shifts the focus onto key principles of criminal procedure and the presumption of innocence in particular. Using Article 6 ECHR and the Directive (EU) 2016/343 as a starting point, it discusses challenges relating to the protective scope of presumption of innocence, the burden of proof rule and the in dubio pro reo principle as core elements of it. Given the transformations law enforcement and criminal proceedings go through in the era of algorithms, big data and artificial intelligence, this article advocates the adoption of specific procedural safeguards that will uphold rule of law requirements, and particularly transparency, fairness and explainability. In doing so, it also takes into account EU legislative initiatives, including the reform of the EU data protection acquis, the E-evidence Proposal, and the Proposal for an EU AI Act. Additionally, it argues in favour of revisiting the protective scope of key fundamental rights, considering, inter alia, the new dimensions suspicion has acquired.
Going beyond the “common suspects”: to be presumed innocent in the era of algorithms, big data and artificial intelligence
This article explores the trend of increasing automation in law enforcement and criminal justice settings through three use cases: predictive policing, machine evidence and recidivism algorithms. The focus lies on artificial-intelligence-driven tools and technologies employed, whether at pre-investigation stages or within criminal proceedings, in order to decode human behaviour and facilitate decision-making as to whom to investigate, arrest, prosecute, and eventually punish. In this context, this article first underlines the existence of a persistent dilemma between the goal of increasing the operational efficiency of police and judicial authorities and that of safeguarding fundamental rights of the affected individuals. Subsequently, it shifts the focus onto key principles of criminal procedure and the presumption of innocence in particular. Using Article 6 ECHR and the Directive (EU) 2016/343 as a starting point, it discusses challenges relating to the protective scope of presumption of innocence, the burden of proof rule and the in dubio pro reo principle as core elements of it. Given the transformations law enforcement and criminal proceedings go through in the era of algorithms, big data and artificial intelligence, this article advocates the adoption of specific procedural safeguards that will uphold rule of law requirements, and particularly transparency, fairness and explainability. In doing so, it also takes into account EU legislative initiatives, including the reform of the EU data protection acquis, the E-evidence Proposal, and the Proposal for an EU AI Act. Additionally, it argues in favour of revisiting the protective scope of key fundamental rights, considering, inter alia, the new dimensions suspicion has acquired.
Automated Justice: Issues, Benefits and Risks in the Use of Artificial Intelligence and Its Algorithms in Access to Justice and Law Enforcement
The use of artificial intelligenceArtificial Intelligence (AI) (AI) in the field of law has generated many hopes. Some have seen it as a way of relieving courts’ congestion, facilitating investigations, and making sentences for certain offences more consistent—and therefore fairer. But while it is true that the work of investigators and judges can be facilitated by these tools, particularly in terms of finding evidenceEvidence during the investigative process, or preparing legal summaries, the panorama of current uses is far from rosy, as it often clashes with the reality of field usage and raises serious questions regarding human rightsHuman rights. This chapter will use the RobodebtRobodebt Case to explore some of the problems with introducing automationAutomation into legal systems with little human oversight. AI—especially if it is poorly designed—has biases in its data and learning pathways which need to be corrected. The infrastructures that carry these tools may fail, introducing novel bias. All these elements are poorly understood by the legal world and can lead to misuse. In this context, there is a need to identify both the users of AIArtificial Intelligence (AI) in the area of law and the uses made of it, as well as a need for transparencyTransparency, the rules and contours of which have yet to be established.
