Iran and China Lead Surge in AI Misuse as OpenAI Bans Surveillance Networks

On 22 Feb­ru­ary 2025, The Hack­er News report­ed that Ope­nAI has banned mul­ti­ple accounts for using Chat­G­PT to cre­ate AI-pow­ered sur­veil­lance tools like­ly orig­i­nat­ing from Chi­na. The arti­cle begins:

Ope­nAI on Fri­day revealed that it banned a set of accounts that used its Chat­G­PT tool to devel­op a sus­pect­ed arti­fi­cial intel­li­gence (AI)-powered sur­veil­lance tool. The social media lis­ten­ing tool is said to like­ly orig­i­nate from Chi­na and is pow­ered by one of Meta’s Lla­ma mod­els, with the accounts in ques­tion using the AI com­pa­ny’s mod­els to gen­er­ate detailed descrip­tions and ana­lyze doc­u­ments for an appa­ra­tus capa­ble of col­lect­ing real-time data and reports about anti-Chi­na protests in the West and shar­ing the insights with Chi­nese author­i­ties. The cam­paign has been code­named Peer Review owing to the “net­work’s behav­ior in pro­mot­ing and review­ing sur­veil­lance tool­ing,” researchers Ben Nim­mo, Albert Zhang, Matthew Richard, and Nathaniel Hart­ley not­ed, adding the tool is designed to ingest and ana­lyze posts and com­ments from plat­forms such as X, Face­book, YouTube, Insta­gram, Telegram, and Red­dit. In one instance flagged by the com­pa­ny, the actors used Chat­G­PT to debug and mod­i­fy source code that’s believed to run the mon­i­tor­ing soft­ware, referred to as “Qianyue Over­seas Pub­lic Opin­ion AI Assis­tant.” Besides using its mod­el as a research tool to sur­face pub­licly avail­able infor­ma­tion about think tanks in the Unit­ed States, and gov­ern­ment offi­cials and politi­cians in coun­tries like Aus­tralia, Cam­bo­dia and the Unit­ed States, the clus­ter has also been found to lever­age Chat­G­PT access to read, trans­late and ana­lyze screen­shots of Eng­lish-lan­guage documents…

Read more

Key Points:

• Ope­nAI iden­ti­fied accounts using Chat­G­PT to devel­op “Qianyue Over­seas Pub­lic Opin­ion AI Assis­tant” for mon­i­tor­ing anti-Chi­na protests globally.
• The com­pa­ny also dis­rupt­ed Iran­ian influ­ence net­works cre­at­ing pro-Pales­tin­ian con­tent for web­sites linked to Inter­na­tion­al Union of Vir­tu­al Media operations.
• Bad actors from Chi­na, Iran, North Korea and Rus­sia increas­ing­ly uti­lize AI tools to enhance var­i­ous phas­es of their dis­in­for­ma­tion campaigns.
• The sur­veil­lance tool was designed to col­lect data from major plat­forms includ­ing X, Face­book, YouTube, Insta­gram, Telegram, and Reddit.