How a fake network pushes pro-China propaganda

How a Chinese marketing network quietly injects political narratives into Taiwanese lifestyle content

A new investigation by Reporters Without Borders (RSF) reveals how certain Facebook pages posing as lifestyle hubs for hobbies and health tips inject Beijing’s political narratives into their feeds to influence public opinion in Taiwan. These pages are tied to the China-based digital marketing company Wubianjie, which blends entertainment, disinformation and political messaging in cognitive warfare campaigns that are difficult to detect.

The spreading of misinformation online

Significance The wide availability of user-provided content in online social media facilitates the aggregation of people around common interests, worldviews, and narratives. However, the World Wide Web is a fruitful environment for the massive diffusion of unverified rumors. In this work, using a massive quantitative analysis of Facebook, we show that information related to distinct narratives––conspiracy theories and scientific news––generates homogeneous and polarized communities (i.e., echo chambers) having similar information consumption patterns. Then, we derive a data-driven percolation model of rumor spreading that demonstrates that homogeneity and polarization are the main determinants for predicting cascades’ size. , The wide availability of user-provided content in online social media facilitates the aggregation of people around common interests, worldviews, and narratives. However, the World Wide Web (WWW) also allows for the rapid dissemination of unsubstantiated rumors and conspiracy theories that often elicit rapid, large, but naive social responses such as the recent case of Jade Helm 15––where a simple military exercise turned out to be perceived as the beginning of a new civil war in the United States. In this work, we address the determinants governing misinformation spreading through a thorough quantitative analysis. In particular, we focus on how Facebook users consume information related to two distinct narratives: scientific and conspiracy news. We find that, although consumers of scientific and conspiracy stories present similar consumption patterns with respect to content, cascade dynamics differ. Selective exposure to content is the primary driver of content diffusion and generates the formation of homogeneous clusters, i.e., “echo chambers.” Indeed, homogeneity appears to be the primary driver for the diffusion of contents and each echo chamber has its own cascade dynamics. Finally, we introduce a data-driven percolation model mimicking rumor spreading and we show that homogeneity and polarization are the main determinants for predicting cascades’ size.

An AI-Generated Content Empire Is Spreading Fake Celebrity Images on Google

A ring of websites using AI and run by an aspiring movie star are behind fake celebrity images featured in Google results, Motherboard found.

How Facebook and Google fund global misinformation

The tech giants are paying millions of dollars to the operators of clickbait pages, bankrolling the deterioration of information ecosystems around the world.

Fake news spreads like an epidemic. Here’s how to stop it.

Researchers study fake news the same way they study the spread of disease. When misinformation “goes viral”, it is not just a metaphor. It is what’s actually happening. via @straits_times str.sg/viz-misinformation

China’s scary lesson to the world: Censoring the Internet works

China is confidently promoting its vision of “Internet Sovereignty.”

As Chinese AI Models Gain Popularity in the West, Their Chatbots Fail to Debunk Pro-China False Claims More Than Half the Time

To download this NewsGuard report, please fill out your details below and you will be redirected to it. If you'd like to learn more about working with NewsGuard, email [email protected].

Facebook Said to Create Censorship Tool to Get Back Into China (Published 2016)

The social network, blocked in China since 2009, has developed software to keep posts from appearing in users’ news feeds in specific geographic areas, current and former employees said.

Google research shows the fast rise of AI-generated misinformation | CBC News

From fake images of war to celebrity hoaxes, AI technology has spawned new forms of reality-warping misinformation online. New analysis co-authored by Google researchers shows just how quickly the problem has grown.

The Internet Coup: A Technical Analysis on How a Chinese Company is Exporting The Great Firewall to Autocratic Regimes | InterSecLab

This research reveals groundbreaking findings on how Geedge Networks is selling an extensive suite of next-generation digital repression tools to client governments around the world.

A well-funded Moscow-based global ‘news’ network has infected Western artificial intelligence tools worldwide with Russian propaganda

An audit found that the 10 leading generative AI tools advanced Moscow’s disinformation goals by repeating false claims from the pro-Kremlin Pravda network 33 percent of the time

How Social Media Rewards Misinformation

A majority of false stories are spread by a small number of frequent users, suggests a new study co-authored by Yale SOM’s Gizem Ceylan. But they can be taught to change their ways.

Tracking Suspicious Facebook Accounts After Takaichi’s Taiwan Remarks

Inauthentic Facebook accounts amplified anti-Japan narratives in Malaysian media after PM Takaichi’s Taiwan contingency remarks.

State media control influences large language models

Millions of people around the world query large language models (LLMs) for information. Although several studies have compellingly documented the persuasive potential of these models1–10, there is limited evidence of who or what influences the models themselves, leading to a flurry of concerns about which companies and governments build and regulate the models. Here we show through six studies that government control of the media across the world already influences the output of LLMs via their training data. We use a cross-national audit to show that LLMs exhibit a stronger pro-government valence in the languages of countries with lower media freedom than in those with higher media freedom. This result is correlational, so to triangulate the specific mechanism of how state media control can influence LLMs, we develop a multi-part case study on China’s media. We demonstrate that media scripted and curated by the Chinese state appears in LLM training datasets. To evaluate the plausible effect of this inclusion, we use an open-weight model to show that additional pretraining on Chinese state-coordinated media generates more positive answers to prompts about Chinese political institutions and leaders. We link this phenomenon to commercial models through two audit studies demonstrating that prompting models in Chinese generates more positive responses about China’s institutions and leaders than do the same queries in English. The combination of influence and persuasive potential across languages suggests the troubling conclusion that states and powerful institutions have increased strategic incentives to leverage media control in the hopes of shaping LLM output.

State media control influences large language models

Millions of people around the world query large language models (LLMs) for information. Although several studies have compellingly documented the persuasive potential of these models1–10, there is limited evidence of who or what influences the models themselves, leading to a flurry of concerns about which companies and governments build and regulate the models. Here we show through six studies that government control of the media across the world already influences the output of LLMs via their training data. We use a cross-national audit to show that LLMs exhibit a stronger pro-government valence in the languages of countries with lower media freedom than in those with higher media freedom. This result is correlational, so to triangulate the specific mechanism of how state media control can influence LLMs, we develop a multi-part case study on China’s media. We demonstrate that media scripted and curated by the Chinese state appears in LLM training datasets. To evaluate the plausible effect of this inclusion, we use an open-weight model to show that additional pretraining on Chinese state-coordinated media generates more positive answers to prompts about Chinese political institutions and leaders. We link this phenomenon to commercial models through two audit studies demonstrating that prompting models in Chinese generates more positive responses about China’s institutions and leaders than do the same queries in English. The combination of influence and persuasive potential across languages suggests the troubling conclusion that states and powerful institutions have increased strategic incentives to leverage media control in the hopes of shaping LLM output.