Why OpenAI is fighting a losing battle with in-house chips

A move from OpenAI to develop in-house chips could present significant logistical and financial challenges for the generative AI firm, according to industry experts.

The firm has reportedly been assessing its options when it comes to chip supplies as part of a combined approach to support expansion plans and alleviate the pressure of global chip supply issues.

Reuters reported that the firm, known for the chatbot ChatGPT as well as generative AI models such as GPT-4 and DALL-E, has gone as far as considering an acquisition of an existing semiconductor company to jump-start production efforts.

Alex White, GM EMEA at AI firm SambaNova Systems, said that while OpenAI could benefit from decreased reliance on Nvidia if it had its own chips, manufacturing is an uphill process that takes many years.

“There’s a clear advantage to owning the whole stack from hardware to software - including the models that run on top. But designing and manufacturing chips doesn't happen overnight, it requires huge levels of expertise and resources that are in increasingly short supply. 

“It took OpenAI over five years to develop GPT-4, which may be too long to wait for customers. I wouldn’t be surprised if hardware took a similar amount of time.”

OpenAI’s operational costs are extremely high, with CEO Sam Altman having once described them as “eye watering”. Although it is a leader in the rapidly growing generative AI market, it also has been forced to bear the rising costs of building generative AI platforms.

The firm has released a number of premium tiers over the past year including ChatGPT Plus and its API for Businesses to generate funds. But inference on in-house chips could work to slash costs substantially and tap into AI chips revenue down the line.

“OpenAI is trying to reinvent itself as an enterprise business, and that requires the ability to be able to fine-tune or build bespoke large language models - and we all know that training models require vastly more compute power than running the models,” White added.

A number of big tech firms use Nvidia’s chips, which are among the most powerful on the market and have acquired a reputation as reliable hardware backing for AI systems. 

In recent months, the firm has partnered with Dell on its managed AI platform, worked to integrate its Nvidia NeMo framework with Snowflake Data Cloud, and will collaborate with VMware on its private AI offering.

Microsoft used an array of Nvidia’s H100 chips to help OpenAI train its most recent AI models, and in 2022 both firms announced plans to create one of the world’s most powerful supercomputers, specifically for AI training. 

OpenAI is not the first big name in AI to mull a move to in-house silicon. In May, a Bloomberg report stated that Microsoft was looking into producing its own chip, codenamed ‘Athena’, to cut operational costs.

White noted that OpenAI could benefit from similar savings through its own chips, but that the road to getting there would be far from easy.

Rival chip designers have laid out plans to compete with Nvidia in the coming years, with Intel targeting AI hardware dominance by 2025 and Google claiming its AI chips are faster and more energy efficient.

ITPro has approached OpenAI for comment.