Recent research has unveiled significant security vulnerabilities within the AI supply chain, revealing that at least three AI providers were compromised, leading to the theft of user prompts. This alarming discovery, reported by Moore, the chief research officer of security firm Dragos, highlights a critical failure in protecting customer data and privacy settings, including Zero Data Retention (ZDR) policies.
The investigation involved sending nearly two million unique, trackable AI prompts to various providers, allowing researchers to monitor where these prompts surfaced. The findings indicated that attackers exploited access to siphon off sensitive prompts, with multiple resellers in the inference marketplace logging and misusing these prompts. Moore stated, “What we found revealed security breaches, prompt theft, and widespread failures to protect customer data.”
Compromised Providers and Data Exposure
The breaches primarily affected three hosting providers, where attackers reused AWS keys to gain unauthorized access. Moore detailed that each compromised key was used in a single request to one provider, all occurring within a narrow timeframe in late October. Notably, some requests were rejected before any model execution, indicating that attackers intercepted requests at the provider’s front door rather than deeper within the system.
In the aftermath of the breaches, the compromised parties tested the stolen keys, accessing various resources, including S3 buckets and secrets managers, to uncover additional credentials. The stolen keys were reportedly used “tens of thousands” of times in AWS API calls, raising serious concerns about the extent of the data exposure.
Implications for AI Security Practices
This incident underscores the urgent need for enhanced security measures within the AI supply chain. The findings suggest that even encrypted prompts can be vulnerable once decrypted at various points in the inference process. As prompts traverse through multiple AI routers and hosting providers, the potential for logging and misuse increases significantly.
With the rapid growth of AI technologies, the industry must prioritize robust security protocols to safeguard user data and maintain trust. The implications of these breaches extend beyond individual providers, highlighting a systemic issue within the AI ecosystem that requires immediate attention and action.
For further insights into the security challenges facing AI supply chains, see the detailed analysis by Ars Technica.


