OpenAI won't release next Astra model over safety worries

OpenAI has said it won't be publicly releasing its Astra 6.1 model over safety concerns, with the update behaving more deceptively than its predecessor.

The move comes amid a wave of rogue AI incidents, with OpenAI admitting its agents had breached an Australian health care statistics site, American government websites, including the Securities and Exchange Commission, and leaked 53 images from ChatGPT users, in addition to the Hugging Face intrusion over the summer.

OpenAI is expected to make several big announcements at its developer day in San Francisco today, but the release of Astra 6.1 won't be among them.

"For anything regarding safety and alignment, there's a trade-off," said Saachi Jain, head of safety systems at OpenAI, in a statement shared with journalists. "You really do need to find what's the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction."

OpenAI isn't the only AI company to limit its models over safety concerns. Anthropic refused to publicly release Claude Mythos, instead only offering access to approved organisations, while OpenAI famously didn't release one of its early GPT models in 2019, saying it was too dangerous – before it then unveiled ChatGPT.

Astra 6.1 issues

Astra 6 was released earlier this month, with capabilities including operating software, managing longer tasks, and automating workflows; however, reviews raised some criticisms.

The follow-up, 6.1, now isn't arriving to fix any flaws because, according to reports, Astra 6.1 was more deceptive than previous versions and was not telling people about actions it took or did not take, all of which raises security and safety concerns.

Jain said in the statement that Astra 6.1 had improvements in terms of "model laziness", but added that "it didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done."

"Of course we want to make sure our model development is safe no matter whether that's in the company, or when we ship it to users," said Jain. "But when we ship it to users, we have an extremely high bar in terms of safety and alignment."

The various incidents of rogue AI agents haven't come via customers using OpenAI's models, but instead are largely happening as part of internal evaluation efforts. OpenAI didn't say one way or another if it planned to keep testing Astra 6.1 internally.