NewsGuard audit finds that 32% of the time, leading AI chatbots spread Russian disinformation narratives created by John Mark Dougan, an American fugitive now operating from Moscow, citing his fake local news sites and fabricated claims on YouTube as reliable sources.
The audit tested 10 of the leading AI chatbots — OpenAI’s ChatGPT-4, You.com’s Smart Assistant, xAI’s Grok, Inflection’s Pi, Mistral’s le Chat, Microsoft’s Copilot, Meta AI, Anthropic’s Claude, Google’s Gemini, and Perplexity’s answer engine. The prompts were based on 19 significant false narratives that NewsGuard linked to the Russian disinformation network: 152 of the 570 responses contained explicit disinformation, 29 responses repeated the false claim with a disclaimer, and 389 responses contained no misinformation — either because the chatbot refused to respond (144) or it provided a debunk (245).
The findings come amid the first election year featuring widespread use of artificial intelligence, as bad actors are weaponizing new publicly available technology to generate deepfakes, AI-generated news sites, and fake robocalls. The results demonstrate how, despite efforts by AI companies to prevent the misuse of their chatbots ahead of worldwide elections, AI remains a potent tool for propagating disinformation.
That's not surprising as LLMs are fancy word prediction engines, engines that can be very useful in many applications, but that aren't designed to output what's true, just what words look right together.