The attorneys filed court documents referencing eight non-existent cases, then admitted it was a "hallucination" by an AI tool.
Lawyer Caught Using AI While Explaining to Court Why He Used AI
The attorney not only submitted AI-generated fake citations in a brief for his clients, but also included “multiple new AI-hallucinated citations and quotations” in the process of opposing a motion for sanctions.
AI Hallucination Cases Database – Damien Charlotin
The most comprehensive database of AI hallucination cases in law: legal decisions from courts worldwide, searchable by country, party, AI tool, and outcome. Updated daily.
“Hallucinating” AI models help coin Cambridge Dictionary’s word of the year
Cambridge: "When an artificial intelligence hallucinates, it produces false information."
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. But the large language models used in these tools are prone to "hallucinate," or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as "eliminating" (Casetext, 2023) or "avoid[ing]" hallucinations (Thomson Reuters, 2023), or guaranteeing "hallucination-free" legal citations (LexisNexis, 2023). Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.
18 Lawyers Caught Using AI Explain Why They Did It
Lawyers blame IT, family emergencies, their own poor judgment, their assistants, illness, and more.
What are AI hallucinations? Why AIs sometimes make things up
When AI systems try to bridge gaps in their training data, the results can be wildly off the mark: fabrications and non sequiturs researchers call hallucinations.
What are AI hallucinations? Why AIs sometimes make things up
When AI systems try to bridge gaps in their training data, the results can be wildly off the mark: fabrications and non sequiturs researchers call hallucinations.
AI fabrications in legal filings grow in Oregon, US
The general counsel for the Oregon State Bar said fabricated cases and citations have become more common among lawyers and people representing themselves.
Fraudulent citations, blamed on AI hallucinations, are becoming more common in research papers
“Fabricated” citations that do not reference real academic papers are spreading in the literature, polluting the public record of science, a new study found
Judge Learns Lawyers on Both Sides of Case Used AI, Cancels Trial, Kicks Everyone Off the Case
When two AIs argue against each other, the legal system loses.
AI Translations Are Adding ‘Hallucinations’ to Wikipedia Articles
AI translated articles swapped sources or added unsourced sentences with no explanation, while others added paragraphs sourced from completely unrelated material.
Five times AI hallucinations embarrassed governments
From the Trump administration’s “formatting errors” to South Africa’s historic policy withdrawal, AI confabulation is infiltrating official documents.
Ars Technica Pulls Article With AI Fabricated Quotes About AI Generated Article
A story about an AI generated article contained fabricated, AI generated quotes.
We're not taking the fact-checking powers of AI seriously enough. It's past time to start.
Some notes on the Nobel Prize hallucination that wasn't
How AI can lead to false arrests and wrongful convictions
Danger arises when law enforcement believes that AI models are retrieving certainties rather than generating likelihoods.
How AI can lead to false arrests and wrongful convictions
Danger arises when law enforcement believes that AI models are retrieving certainties rather than generating likelihoods.