"Prohibitive" generative AI costs could pose challenges for firms in 2024, analysts warn

A CGI image of a white cube made of glowing energy slotting into a blocky, dark grey computer motherboard to represent generative AI.
(Image credit: Getty Images)

Some businesses will struggle to adopt generative AI in 2024 due to untenable costs of deployment, according to a new report.

CCS Insight has predicted a “cold shower” for generative AI in 2024 following a surge in investment over the past year that has been likened to a ‘gold rush’

Ben Wood, chief analyst and CMO at CCS Insight told ITPro that while a “bullish” outlook on generative AI remains due to a growing interest among enterprises, the technology will also present a slew of new challenges.  

“What we feel is the hype in 2023 has really got to the point of being a short-term great amount of excitement around this, but the commercial realities of deploying generative AI at scale are extremely concerning,” said Wood.

“The investment you need, in order to scale it, in order to add the massive computing power that you need to deliver generative AI in the cloud and then get it to devices, is going to be prohibitively expensive for some smaller companies and developers.

“So the big hyperscalers, the Googles, the Microsofts, and the Metas of the world have the resources that they can scale that up, and they can keep funding that kind of investment, but for smaller companies is going to be harder.”

Growing generative AI demands

As models have become larger and demands from businesses for more fine-tuned, customizable models have increased. However, estimates for the cost of training and implementing AI have also risen

Even OpenAI, which has been the recipient of billions of dollars in Microsoft funding, has struggled with operating costs and is reportedly looking to produce its own chips to meet up with workload demand.

Wood also highlighted the need for companies to keep safety measures on AI up-to-date, and to stay in line with brewing legal and regulatory requirements on AI. 

As part of its prediction on the rocky road for generative AI in 2024, CCS Insight noted that many firms would slow uptake of the technology to properly assess its risks and evaluate whether they outweigh its benefits for their use cases.


Whitepaper from Dell on their world-record performance for AL and ML with image of metal sculpture from the ground up

(Image credit: Dell)

World-record performance for AI and ML

Discover how quickly data can be collected, processed, and accessed when stored on Dell PowerEdge servers.


This is reflected in another of the report’s predictions, which sets out the expectation that more large organizations will set up AI oversight committees in 2024 to help shape ethical AI, reduce algorithmic bias, and meet legal obligations.

Companies are expected to appoint a diverse mix of industry professionals and academics, combining STEM and the humanities, to address potential risks. Truly diverse committees can work to prevent AI systems from engaging in discriminatory practices and to specifically reduce ableism, racism, and other harmful biases.

In September, Gartner reported ramped-up investment in application security as businesses sought to mitigate the risks associated with generative AI, including data leaks and ‘hallucinations’ — a term used to describe instances in which an AI model insists incorrect facts or statements to be true.

On potential hope in the sector, Wood acknowledged the growing field of hybrid AI, in which foundation models such as LLMs are run partly on-premises and partly on the cloud, to balance performance and compute cost.

Dell has invested heavily in providing on-prem AI for its customers, and models such as Meta’s Llama 2 can be run locally if one has a suitably powerful graphics processing unit (GPU). But Wood cautioned that in the short term, companies could struggle to make the hybrid model work.

The predictions were numbered among a total of 85 in CCS Insights' 2024 & Beyond - Predictions report, covering tech-driven changes across the areas of enterprise, networks, devices, and connectivity between 2024 and 2030. 

Technologies such as generative AI, 6G, and satellite connectivity were featured heavily throughout the report, alongside predictions for established technologies such as 5G and the Internet of Things.

Rory Bathgate
Features and Multimedia Editor

Rory Bathgate is Features and Multimedia Editor at ITPro, overseeing all in-depth content and case studies. He can also be found co-hosting the ITPro Podcast with Jane McCallion, swapping a keyboard for a microphone to discuss the latest learnings with thought leaders from across the tech sector.

In his free time, Rory enjoys photography, video editing, and good science fiction. After graduating from the University of Kent with a BA in English and American Literature, Rory undertook an MA in Eighteenth-Century Studies at King’s College London. He joined ITPro in 2022 as a graduate, following four years in student journalism. You can contact Rory at rory.bathgate@futurenet.com or on LinkedIn.