On 22 February 2025, The Hacker News reported that OpenAI has banned multiple accounts for using ChatGPT to create AI-powered surveillance tools likely originating from China. The article begins:
OpenAI on Friday revealed that it banned a set of accounts that used its ChatGPT tool to develop a suspected artificial intelligence (AI)-powered surveillance tool. The social media listening tool is said to likely originate from China and is powered by one of Meta’s Llama models, with the accounts in question using the AI company’s models to generate detailed descriptions and analyze documents for an apparatus capable of collecting real-time data and reports about anti-China protests in the West and sharing the insights with Chinese authorities. The campaign has been codenamed Peer Review owing to the “network’s behavior in promoting and reviewing surveillance tooling,” researchers Ben Nimmo, Albert Zhang, Matthew Richard, and Nathaniel Hartley noted, adding the tool is designed to ingest and analyze posts and comments from platforms such as X, Facebook, YouTube, Instagram, Telegram, and Reddit. In one instance flagged by the company, the actors used ChatGPT to debug and modify source code that’s believed to run the monitoring software, referred to as “Qianyue Overseas Public Opinion AI Assistant.” Besides using its model as a research tool to surface publicly available information about think tanks in the United States, and government officials and politicians in countries like Australia, Cambodia and the United States, the cluster has also been found to leverage ChatGPT access to read, translate and analyze screenshots of English-language documents…
Key Points:
• OpenAI identified accounts using ChatGPT to develop “Qianyue Overseas Public Opinion AI Assistant” for monitoring anti-China protests globally.
• The company also disrupted Iranian influence networks creating pro-Palestinian content for websites linked to International Union of Virtual Media operations.
• Bad actors from China, Iran, North Korea and Russia increasingly utilize AI tools to enhance various phases of their disinformation campaigns.
• The surveillance tool was designed to collect data from major platforms including X, Facebook, YouTube, Instagram, Telegram, and Reddit.