Licensing is an important component of the re-usability of research data, itself part of the FAIR principles: without clear, machine-readable licensing, datasets risk becoming technically...
A large-scale audit of dataset licensing and attribution in AI
The race to train language models on vast, diverse and inconsistently documented datasets raises pressing legal and ethical concerns. To improve data transparency and understanding, we convene a multi-disciplinary effort between legal and machine learning experts to systematically audit and trace more than 1,800 text datasets. We develop tools and standards to trace the lineage of these datasets, including their source, creators, licences and subsequent use. Our landscape analysis highlights sharp divides in the composition and focus of data licenced for commercial use. Important categories including low-resource languages, creative tasks and new synthetic data all tend to be restrictively licenced. We observe frequent miscategorization of licences on popular dataset hosting sites, with licence omission rates of more than 70% and error rates of more than 50%. This highlights a crisis in misattribution and informed use of popular datasets driving many recent breakthroughs. Our analysis of data sources also explains the application of copyright law and fair use to finetuning data. As a contribution to continuing improvements in dataset transparency and responsible use, we release our audit, with an interactive user interface, the Data Provenance Explorer, to enable practitioners to trace and filter on data provenance for the most popular finetuning data collections: www.dataprovenance.org.
FAIR Principles - GO FAIR
In 2016, the ‘FAIR Guiding Principles for scientific data management and stewardship’ were published in Scientific Data. The authors intended to provide guidelines to improve the Findability, Accessibility, Interoperability, and Reuse of digital assets. The principles emphasise machine-actionability (i.e., the capacity of… Continue reading →
Creative commons licenses and copyright may not stop academic work being used to train AI - Impact of Social Sciences
Considering the legal standing of creative commons licenses & copyright, Martin Eve suggests legal protections for academic work are unlikely to be forthcoming.
FAIRdata.ai — FAIR Data Assessment
Assess your research data's FAIRness. Automated pipeline using F-UJI + Claude AI. Free to use.
DataLicenses.org
Machine-readable hints for AI agents/crawlers; easy to adopt, rely on compliance.
Permissive-Washing in the Open AI Supply Chain: A Large-Scale Audit of License Integrity
Permissive licenses like MIT, Apache-2.0, and BSD-3-Clause dominate open-source AI, signaling that artifacts like models, datasets, and code can be freely used, modified, and redistributed. However, these licenses carry mandatory requirements: include the full license text, provide a copyright notice, and preserve upstream attribution, that remain unverified at scale. Failure to meet these conditions can place reuse outside the scope of the license, effectively leaving AI artifacts under default copyright for those uses and exposing downstream users to litigation. We call this phenomenon ``permissive washing'': labeling AI artifacts as free to use, while omitting the legal documentation required to make that label actionable. To assess how widespread permissive washing is in the AI supply chain, we empirically audit 124,278 dataset $\rightarrow$ model $\rightarrow$ application supply chains, spanning 3,338 datasets, 6,664 models, and 28,516 applications across Hugging Face and GitHub. We find that an astonishing 96.5\% of datasets and 95.8\% of models lack the required license text, only 2.3\% of datasets and 3.2\% of models satisfy both license text and copyright requirements, and even when upstream artifacts provide complete licensing evidence, attribution rarely propagates downstream: only 27.59\% of models preserve compliant dataset notices and only 5.75\% of applications preserve compliant model notices (with just 6.38\% preserving any linked upstream notice). Practitioners cannot assume permissive labels confer the rights they claim: license files and notices, not metadata, are the source of legal truth. To support future research, we release our full audit dataset and reproducible pipeline.
A Collectivist, Economic Perspective on AI
Information technology is in the midst of a revolution in which omnipresent data collection and machine learning are impacting the human world as never before. The word ``intelligence'' is being...
LICENSE · moonshotai/Kimi-K3 at 2496450e92e425c886db095102a52a6682ca3970
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
LICENSE · moonshotai/Kimi-K3 at main
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Find Open Datasets for AI and Research | Kaggle
Browse and download hundreds of thousands of open datasets for AI research, model training, and analysis. Join a community of millions of researchers, developers, and builders to share and collaborate on Kaggle.
LICENSE · MiniMaxAI/MiniMax-H3 at 73372e6cf53e414edd3ab03e357717fb0602e758
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Unlawful by design: Exposing the human rights costs of generative AI - Amnesty International
This briefing examines how standalone generative AI systems, based on unlawful web scraping, are in conflict with international human rights law (IHRL) and standards through their design, development and deployment. While these technologies promise sophisticated automation and efficiency, they rely on data collection and model training practices that abuse privacy rights, enable discrimination, and threaten […]
The AI "Evaluation Crisis" Is an Opportunity to Get Data Flow Right
Why the AI evaluation crisis could force a reckoning on dataset provenance, attribution, and consent.
Firms like Meta and A16z admit having to pay billions for training data would ruin their generative-AI plans as they fight new copyright rules
Meta, Google, Microsoft, and Andreessen Horowitz are trying to keep AI developers from having to pay for copyrighted material used in AI training.
RSL: Really Simple Licensing
The open content licensing standard for the AI-first Internet
Licensing terms for data in the Atmosphere
I’d like to raise a fun and interesting topic, and that’s intellectual property and content licensing. The data that flies around the Atmosphere is all visible and public. We’re at a stage within the Atmosphere where services are beginning to grow, and we’re beginning to see lexicons and data that express creative works that extend in length beyond microblogging. I think that the Standard.site lexicons are a great example. But these creations lead to further questions. Who has permission to...
