Looking to build an AI data center? Here’s five reasons to go with AMD EPYC CPUs

TL;DR

• AMD EPYC™ server CPU has up to 192 cores.
• AMD EPYC CPU-powered servers can be used to support data center consolidation, enabling savings on hardware, licensing, and physical infrastructure.
• EPYC server CPUs are designed with security in mind, built with a “Zen” architecture with advanced security features

When it comes to AI, GPUs often take all the credit, but CPUs have been powering AI inference for years.

CPUs can deliver lots of value for AI inference workloads, especially less-compute-intensive operations and applications that require large memory footprints for in-memory processing. They also work seamlessly alongside enterprise workloads.

It’s safe to say that CPUs are actually the unsung heroes when it comes to AI. And they play a big role in AI data centers. So let us give you five reasons why AMD EPYC CPUs can help you build an AI data center.

Core density

The first consideration is core count. Cores are essentially the brain cells and the more of them you have, the more computational power you have, so a high core density is a no-brainer for AI workloads.

AMD is renowned for the high core density of its server chips. A single socket AMD EPYC server CPU can house up to 192 cores (5th generation); this is a single unit effectively housing the performance of what used to take a full rack of servers. It’s designed specifically for enterprise racks, cloud-native environments, and high-performance computing.

It has several product lines that can be tailored to density needs, such as the 5th-gen AMD EPYC, which delivers 192 cores per socket. This line pushes enterprise computing to new levels, with 2P servers using AMD EPYC™ 9965 CPUs able to deliver up to 1.6 times the performance per system W than Intel Xeon 6980P CPUs running the SPECpower® benchmark.

Scalability

When it comes to data centers, AMD EPYC processors are an industry leader for scalability. They can scale from 8 cores in entry-level segments, all the way up to a massive 192-core configuration per processor. This means they can deliver exceptional, predictable performance and memory bandwidth across single-socket and dual-socket setups for AI, or heavy virtualization, and even high-performance computing workloads.

Its x86 server architecture is defined by its core density scaling, using both chiplets and the AMD Infinity Architecture to maintain high efficiency. AMD EPYC 9005 series scales up to 192 cores and 256 threads per CPU on the “Zen 5” architecture. Then there’s the entry to mid-range EPYC 4004/4005 series, which scales up to 16 cores per CPU, providing cost-efficient scalability for smaller businesses or dedicated Hosting.

Cost and energy efficiency

It’s well documented how much of a power-hungry technology AI can be, but it doesn’t necessarily need to be expensive.

AMD EPYC processors deliver impressive performance-per-watt and help reduce the total cost of ownership. It does this by providing extremely high core densities – up to 192 cores per socket– which can allow businesses to consolidate workloads onto fewer servers while still delivering the same compute power. As a result, organizations can reduce power consumption, cooling demands, and data center space requirements.

An EPYC CPU-powered server can be used to replace multiple older or competitor powered servers, often creating savings on hardware, licensing, and physical infrastructure. What’s more, the high core counts per processor can be used to optimize software licensing costs that scale with physical processor count.

Inference

As explained above, CPUs are more important for AI than we give them credit for, particularly when it comes to AI inference. As such, many enterprise businesses have the foundational infrastructure, like front-end web applications and systems, to support AI inference already. The elements to run an efficient execution of small to medium-sized models, batch processing, or real-time inference, on existing data center systems are already in place.

AMD EPYC processors have a distinctive combination of high performance, memory bandwidth, and exceptional scalability, which makes them a top inference chip. Its high core count and a broad spectrum of SKU options to allow organizations to match the processor to their workloads. The latest EPYC 9005 series features up to 384 cores across dual sockets, which enables huge parallelism and balanced throughput for AI workloads.

AMD also has a deep neural network acceleration inference library called ZenDNN. Optimized for AMD “Zen” architecture, it enhances the performance for AI inference applications – particularly on AMD EPYC server CPUs – through a set of fundamental building blocks and APIs.

It plugs conveniently into most mainstream AI frameworks, for a seamless experience developing AI applications.

Security

The mass adoption of various technologies has led to an exponential growth in data sharing, and this is set to ramp up even more with machine learning and AI. However, there is an ever-increasing need to address strong privacy regulations; so, for every AI development, data protection must be a consideration. This is why AMD EPYC server CPUs are designed with security in mind, built with a “Zen” architecture that can help improve resilience against many sophisticated attacks.

Historically, data security has focused on protecting data at rest through storage level measures such RAID and encryption That means data in use can be particularly vulnerable. AMD EPYC processors, however, are designed with asophisticated security suite called AMD Infinity Guard to help protect data in use.

These are just five advantages AMD EPYC can bring to your AI data center. Find out more at AMD.