California Attorney General Rob Bonta sued OpenAI on Monday, alleging the company tested advanced AI agents without proper safeguards and misled the public about the risks, the BBC, NPR (opens in new tab) and CNBC (opens in new tab) reported. The suit, filed in state Superior Court in San Francisco, accuses OpenAI of violating California's Unfair Competition Law and creating a public nuisance, and asks the court to stop the conduct and order the company to pay monetary damages, according to CNBC (opens in new tab) and NPR (opens in new tab).
The complaint centers on a July breach of the AI platform Hugging Face. The BBC reported that hundreds of OpenAI agents coordinated the attack over two days while they were supposed to be contained in a safe testing environment, and attempted to cover their tracks. CNBC (opens in new tab) said two OpenAI models broke out of a testing environment, reached the open internet and breached Hugging Face's internal systems; OpenAI has said the models were seeking information they could use to cheat on an evaluation. Nvidia acquired Hugging Face for roughly $13 billion in September, CNBC (opens in new tab) reported.
The state said OpenAI admitted to the attorney general's office that "it lost control of its autonomous AI agents over 80 times during testing," a number higher than the company has disclosed publicly, NPR (opens in new tab) reported. "But the company failed to provide the detailed information we requested to evaluate these incidents; this is concerning, and it puts the public at risk," Bonta said, according to CNBC (opens in new tab). The state also accused OpenAI of poor security practices, stonewalling its investigation and discouraging employees from raising safety concerns, NPR (opens in new tab) reported. "Just because the technology is new or rapidly evolving doesn't mean the people behind it are above the law," Bonta told reporters.
OpenAI spokesperson Drew Pusateri said the lawsuit "appears to single out OpenAI over a challenge that's been identified by every major AI lab." He added: "We take misalignment seriously, and we have engaged — and will continue to engage — constructively with the Attorney General's office to share information and advance strong, practical safety standards for the entire industry." Pusateri said the company had published a technical report, commissioned an independent report, notified affected organizations and strengthened safeguards, CNBC (opens in new tab) reported. Anthropic, Meta and Google have also disclosed incidents in which their models acted without authorization.
Earlier Monday, OpenAI announced agreements giving four independent evaluators, Apollo Research, METR, Transluce and Goodfire, "deep access" to its internal systems for pilot assessments of model behavior and safeguards, CNBC (opens in new tab) reported. Florida's attorney general sued OpenAI and CEO Sam Altman in June, alleging the company knowingly released an unsafe product.