Dreamforce 2026 showed AI safety is the new big tech battleground

AI safety risks were the big talking point on the opening day of Dreamforce 2026, with the annual conference’s star-studded line-up bringing the topic to the fore.

Anthropic CEO Dario Amodei joined Salesforce chief Marc Benioff during the opening keynote to discuss the company’s Claudeforce launch. Recent comments on the concerning pace of AI development dominated the discussion, however.

With companies such as Anthropic rolling out increasingly powerful models, Amodei told Benioff that the industry needs to have a candid discussion about development timelines, AI safety, and the introduction of more robust guardrails.

“We can always be better, let’s make our practices better, let’s recommit to transparency,” he told the Salesforce CEO. “Let’s invest more in safety, then let’s organize the rest of the industry and say ‘what can we do to set standards for everyone?’.”

Amodei’s comments came after he published a lengthy essay on 12 September arguing that the AI industry needs to slow down the pace of AI development to counter potential harms. This call for “pacing the frontier” was echoed by OpenAI CEO Sam Altman and xAI chief Elon Musk online.

Why the focus on AI safety?

The topic of AI safety has been thrust into the spotlight after a series of high profile security incidents involving agents at OpenAI, Anthropic, and Meta. These incidents saw agents escape testing environments and breach third-party companies such as Hugging Face.

Combined with whistle blower claims that AI could severely harm humanity in recent days, the topic appears to have reached boiling point.

Speaking during a fireside chat with Benioff later in the day, Altman said the Hugging Face debacle “triggered a real rest, not just in our own company, but in the world”.

“I think people also feel the speed with which things are moving,” he told Benioff. “There have been things like the Hugging Face accidents, other accidents in the field.”

Altman added that he is confident both in the ability of the industry and OpenAI itself to slow down the pace of development and sharpen the focus on AI safety.

Individual responsibility

While Altman and Amodei have called for an international effort to slow down the pace of AI development, Nvidia CEO Jensen Huang thinks the responsibility rests firmly on individual companies – not erecting new barriers.

“We don’t need any new laws, we don’t need any new regulations,” he told Benioff.

Huang is by no means calling for a Wild West approach to AI development, rather arguing that the onus must be placed on developers that are rolling out products they feel they cannot adequately contain.

“Safety is paramount,” he said. “Safety is an engineering problem. We’re developing software after all, we’re developing computing systems after all. It’s complicated computing systems, but ultimately it’s a computing system.”

Huang said that testing environments need to be “built in a good way” to avoid repeats of the incidents involving Meta, OpenAI, and Anthropic. That responsibility lies with them, not by insisting the rest of the industry adheres to their playbook.

As ITPro reported in August, all three companies involved in ‘rogue AI’ incidents worked with the same testing company before agents escaped their sandbox environments.

“If we're not confident about the safety of the products, like all companies … if you build a product or a service and you're not confident in its functionality, capability, or safety, then don't release it,” he said.

“That's a very obvious thing to do. You pace yourself until you are confident you're releasing something that the market would appreciate.”

FOLLOW US ON SOCIAL MEDIA

Follow ITPro on Google News and add us as a preferred source to keep tabs on all our latest news, analysis, views, and reviews.

You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.