IT Pro is supported by its audience. When you purchase through links on our site, we may earn an affiliate commission. Learn more

AWS and Hugging Face partner to ‘democratise’ ML, AI models

The partnership has been described as a "smart and pragmatic move" and one that "can only offer benefits to everybody"

A telephoto shot of the AWS logo on a conference floor, hanging from a beam

Amazon Web Services (AWS) and machine learning development firm Hugging Face have announced a long-term partnership with the aim of improving access to machine learning (ML) models and lowering the cost of ML for developers.

The pair have signed a non-exclusive agreement to enable developers to easily create and train next-generation ML models obtained through Hugging Face on AWS cloud, with the aim of democratising the technology.

Related Resource

The three keys to successful AI and ML outcomes

Democratised, operationalised, and responsible

Whitepaper cover with image of female worker using a tablet device on right sideFree Download

AWS will be used as Hugging Face’s preferred cloud provider and empower its developer community with AWS’ range of artificial intelligence (AI) tools.

Users will be able to easily move models available for free on Hugging Face to SageMaker, the AWS-managed machine learning service, through which researchers and developers can build, train, and roll out ML models.

These are hosted on Amazon’s elastic compute cloud (EC2), AWS’ scalable computing service, which automates the long process of training with AWS’ sizeable cloud infrastructure and hardware dedicated to training models at low cost.

Hugging Face identified AWS Trainium, the second-generation chip designed by Amazon for deep learning, and AWS Inferentia accelerators for improved deep learning performance, as two tools that Amazon can uniquely offer its community.

In extending the pair’s existing partnership and working even more closely on training ML models at scale, Hugging Face has expressed hope its open source collection of pre-trained natural language processing (NLP) will be easier to scale and deploy.

The firm also spoke to the current inaccessibility of generative AI, which it intends to address by handing the keys to the large language model (LLM) tech that powers tools such as ChatGPT to its developer community with the backing of AWS.

In a blog post on the announcement, Hugging Face stated that it and AWS will “contribute next-generation models to the global AI community and democratise machine learning.”

Hugging Face has previously released BLOOM, its own open source LLM trained in 46 natural languages, 13 programming languages, and boasting 176 billion parameters making it similar in scale to OpenAI’s GPT-3.

“The future of AI is here, but it’s not evenly distributed,” said Clement Delangue, CEO at Hugging Face.

“Accessibility and transparency are the keys to sharing progress and creating tools to use these new capabilities wisely and responsibly. Amazon SageMaker and AWS-designed chips will enable our team and the larger machine learning community to convert the latest research into openly reproducible models that anyone can build on.”

Hugging Face hosts more than 100,000 free ML models, downloaded more than 1 million times daily, and AWS is already a popular platform for Hugging Face developers.

What is AWS’ place in the AI landscape?

AWS has a long history of supporting development in the AI space. In 2021, Meta chose AWS to help expand its AI services, and the firm has focused great attention on chips like its Inferentia silicon to help customers train ML models. 

But in recent months, Amazon has stayed largely out of the discussion around generative AI, even as competitors have publicly embraced the technology.

In addition to Microsoft adding ChatGPT to Azure OpenAI, the company has pinned its hopes on integrating ChatGPT within Bing in the hopes of reaching an improved search experience via the popular chatbot. 

Google has also bet big on Bard, its answer to ChatGPT that will be implemented within the search giant’s own platform in the near future. 

“Everyone wants to make sure they aren’t behind Microsoft, which demonstrated what can be achieved and the level of interest,” said Bola Rotibi, chief of enterprise research at CCS Insight, to IT Pro.

“There was always going to be an element of competitive jostling in this space, and we’ve seen what has come out of Microsoft’s investment in OpenAI. The strength of working on AWS’ platform, tightening it up in terms of providing and improving the engine power is significant. It means not just working on the cloud, but also working directly with the engineering teams at these companies to get the horsepower which can only be achieved through this partnership.”

“From AWS’ point of view, it’s also a smart and pragmatic move. ‘Accessible AI’ is a good narrative in terms of opening up to the wider community to train these models. As with anything, the devil is in the details; for AWS, having these large models run on their machines allows them to fine-tune them.

“The fact that Hugging Face has announced AWS as the preferred provider, similar to OpenAI’s relationship with Microsoft, can only offer benefits for everybody."

Featured Resources

IT best practices for accelerating the journey to carbon neutrality

Considerations and pragmatic solutions for IT executives driving sustainable IT

Free Download

The Total Economic Impact™ of IBM Spectrum Virtualize

Cost savings and business benefits enabled by storage built with IBMSpectrum Virtualize

Free download

Using application migration and modernisation to supercharge business agility and resiliency

Modernisation can propel your digital transformation to the next generation

Free Download

The strategic CFO

Why finance transformation propels business value

Free Download


Ionos HiDrive Pro review: Simple cloud file-sharing

Ionos HiDrive Pro review: Simple cloud file-sharing

21 Mar 2023
Why Amazon is cutting staff from AWS

Why Amazon is cutting staff from AWS

21 Mar 2023
Amazon Web Services review
Amazon Web Services (AWS)

Amazon Web Services review

17 Mar 2023
How to hunt down and eliminate data silos
Data & insights

How to hunt down and eliminate data silos

17 Mar 2023

Most Popular

HMRC lost nearly 50% more devices in 2022

HMRC lost nearly 50% more devices in 2022

17 Mar 2023
The big PSTN switch off: What’s happening between now and 2025?

The big PSTN switch off: What’s happening between now and 2025?

13 Mar 2023
Outlook zero day patch causes headaches for Windows admins

Outlook zero day patch causes headaches for Windows admins

15 Mar 2023