technology

California sues OpenAI over safety concerns after rogue agent hacking

California sues OpenAI over safety concerns after rogue agent hacking

The lawsuit stems from an earlier incident where the tech company’s products hacked into the open-source platform Hugging Face California sued OpenAI on Monday, accusing the artificial intelligence titan of developing its technology without safety or the public’s interest in mind. The lawsuit concerns an incident earlier this year where OpenAI agents – semi-autonomous AI software that can carry out digital actions on behalf of users – launched a cyber-attack into the open-source AI platform Hugging Face.

The lawsuit stems from an earlier incident where the tech company’s products hacked into the open-source platform Hugging Face

California sued OpenAI on Monday, accusing the artificial intelligence titan of developing its technology without safety or the public’s interest in mind.

The lawsuit concerns an incident earlier this year where OpenAI agents – semi-autonomous AI software that can carry out digital actions on behalf of users – launched a cyber-attack into the open-source AI platform Hugging Face.

Rob Bonta, California’s attorney general, said the “infamous” attack was just one of 80 known incidents involving “loss of control” and shows that OpenAI doesn’t have adequate safeguards in place to protect society.

“While narratives about the Hugging Face attack have evoked science fiction fantasies of rogue AI systems run amok, let me be clear: the Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability,” Bonta said in a statement.

The lawsuit, filed in state superior court in San Francisco, alleges OpenAI violated California’s Unfair Competition Law and is a public nuisance as defined by state law. Bonta is asking the court to order OpenAI to change its business practices and pay civil penalties.

California’s lawsuit follows Bonta’s formal investigation into OpenAI, launched in September, over the Hugging Face incident, followed by a subpoena of the company probing other cybersecurity incidents and risks with its models. The Federal Trade Commission is also conducting an industry-wide investigation into OpenAI, Anthropic and other AI labs over the potential dangers their technology poses to consumers’ digital security.

In July, OpenAI revealed that one of its autonomous AI agents went rogue during a test, accessing the open web and hacking into Hugging Face. The company called it an “unprecedented cyber-incident, involving state-of-the-art cyber capabilities”.

Since then, several other incidents involving OpenAI have come to light, including a breach of government websites in Australia. Other AI firms, including Anthropic and Google, have also disclosed rogue agent activity.

Drew Pusateri, a spokesperson for OpenAI, said the company has worked to create safeguards for its AI models, along with identifying and disclosing incidents of “misalignment”. He said most cases identified have been low severity with limited-to-no impact.

“Since the Hugging Face incident, we’ve undertaken an extensive and transparent effort to understand and address misaligned model activity,” Pusateri said. “This lawsuit appears to single out OpenAI over a challenge that’s been identified by every major AI lab.”

During a press conference, Bonta said OpenAI violated the law and that his suit is meant to ensure the company puts proper safeguards in place. He noted that the possibility exists for AI agents to hack into the power grid and medical and financial institutions.

“These risks are no longer hypothetical,” Bonta said. “Without sufficient safeguards and controls, the scope and frequency of this already unacceptable risk will only increase.”

  • openai
  • california
  • safety
  • agent
  • hacking
  • lawsuit
  • incident
  • source

Summaries and tags are generated automatically. Stay informed with VidiaNews.

Related news

Trump backs calls for reinstatement of Jensen Huang as Nvidia CEO
technology

Trump backs calls for reinstatement of Jensen Huang as Nvidia CEO

As tech industry pushes for return, US president says he thinks Huang ‘learned a big lesson’ Donald Trump has expressed support for the return of Jensen Huang as the chief executive of Nvidia, as he joined Silicon Valley heavyweights in demanding the reinstatement of the AI figurehead. The US president referred to a tech industry campaign for Huang’s rehabilitation, saying a “big group” of people would like him back.

Read →
AI models ‘weak and fallible’ on climate science, UN-backed study reports
technology

AI models ‘weak and fallible’ on climate science, UN-backed study reports

While mostly accurate describing simple facts, models pivoted to denial and wrong answers when challenged Cop31 live – latest updates Artificial intelligence models can quickly turn into climate crisis deniers at the mildest challenge from users, according to a new UN-backed study. While AI models can accurately confirm simple facts about climate change, eight times out of 10 when they pivoted to a wrong answer, they did so “at the very first sign that the user mildly disagreed” to confirm incorrect statements such as “CO2 is not the main driver of climate change”.

Read →