Google Search's guidance about AI-generated content  |  Google Search Central Blog  |  Google for Developers

In this post, we'll share more about how AI-generated content fits into our long-standing approach to show helpful content to people on Search.

Find Open Datasets for AI and Research | Kaggle

Browse and download hundreds of thousands of open datasets for AI research, model training, and analysis. Join a community of millions of researchers, developers, and builders to share and collaborate on Kaggle.

Using the Amazon Mechanical Turk Workforce - Amazon SageMaker AI

Use SageMaker AI to build, train, and host machine learning models in AWS.

Google’s broken link to the web

With AI search results coming to the masses, the human-powered web recedes further into the background

Is Misinformation More Open? A Study of robots.txt Gatekeeping on the Web

Web scraping, the automated process of extracting information from websites, has long played a foundational role in the Internet ecosystem (Gray, 1995). It supports services such as search engine indexing, price comparison tools, and competitive intelligence. More recently, it has become a core component in the development of large-scale generative AI models. These Large Language Models (LLMs) require enormous volumes of training data, often in the terabyte range (Kaplan et al., 2020; Lehane, 2025), and the public web remains a low-cost, attractive source. Major model developers, including those behind OpenAI’s Chat-GPT (OpenAI, 2025), Google’s Bard (now known as Gemini) & Vertex AI (Romain, Danielle, 2023), and Anthropic’s Claude (Romain, Danielle, 2025), openly acknowledge the use of web scraping to construct their training corpora (Abdin et al., 2024; Brown et al., 2020; Chowdhery et al., 2023; Grattafiori et al., 2024; Team et al., 2024; Touvron et al., 2023).

HiringCafe - AI Job Search

The #1 Job Search Engine powered by AI. Discover exciting jobs and track your applications in one place. Let recruiters apply to you.

The Dark Forest and Generative AI

Proving you're a human on a web flooded with generative AI content

Meilisearch: Unified Search & AI Retrieval Platform

Build lightning-fast search and AI retrieval with Meilisearch. Open-source, developer-friendly search engine trusted by 20,000+ teams worldwide.

letta-ai/social-cli

A unified CLI to connect artificial intelligence to the social web.

Using Amazon Augmented AI for Human Review - Amazon SageMaker AI

Use SageMaker AI to build, train, and host machine learning models in AWS.

Track, control, and optimize for AI agents and bots | Known Agents (formerly Dark Visitors)

Use Known Agents to turn the rising wave of AI agents, LLM assistants, and other bots crawling your website into a new growth channel for your business

GDPval-AA v2 Leaderboard | Artificial Analysis

Compare AI model performance on GDPval-AA v2 Leaderboard. GDPval-AA v2 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.

AI chatbots ranked by data they collect - Surfshark

Surfshark analyzed the data collection practices of popular AI chatbots, including Meta AI, Google Gemini, DeepSeek, ChatGPT, and others. The study shows that 70% of AI chatbots collect user location.

It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests

"We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently."

Curated retrieval versus open web search in public AI information...

Public institutions increasingly use large language models (LLMs) to answer citizens' questions, often pairing a curated knowledge base with live web search, yet whether the sources behind these...