OpenAI has reported the disruption of a “coordinated campaign” aimed at extracting reasoning capabilities from its AI models, attributing the activity to a Chinese competitor, Moonshot AI. The company first detected suspicious behavior on July 1, which escalated to 16,000 prompts from 4,000 users by July 24 and 25, ultimately reaching 15,000 suspicious users by July 28, when OpenAI claimed to have fully disrupted the operation.
The method employed by the attackers was described by OpenAI as “novel.” They allegedly copied encrypted reasoning data from one conversation and then prompted the model in a separate conversation to decrypt and transcribe that content into plain text. OpenAI clarified that the attackers did not breach encryption or gain direct access to user conversations but instead manipulated model interactions to reproduce protected reasoning in a manner that violated the company’s terms of service.
While OpenAI has not provided specific technical evidence for its attribution, it indicated that individuals associated with Moonshot AI were behind a “core cluster” of the suspicious activity. This attribution aligns with previous accusations from American AI companies and the U.S. government, which have claimed that Chinese firms like Moonshot AI engage in systematic distillation attacks on U.S. AI models. Cybersecurity experts have noted that these companies often utilize black or gray markets to acquire numerous accounts for models such as Claude and ChatGPT, subsequently inundating these models with prompts to replicate their capabilities.
OpenAI has opted not to disclose further details for security reasons but has communicated the incident to organizations like the Frontier Model Forum. In response to the attack, OpenAI has banned offending accounts, enhanced signup and infrastructure controls, expanded network monitoring, and addressed a vulnerability that allowed users to transfer encrypted data between conversations.
As the landscape of AI continues to evolve, the implications of such attacks raise significant concerns regarding the security of proprietary models and the ongoing competition in the AI sector.
For more details, see CyberScoop.
