Is AMD EPYC the right CPU for your AI models?

TL;DR

• AMD EPYC CPUs are powerful enough to run many AI models without the need for GPUs
• AMD EPYC CPUs are validated by manufacturers including HPE, Lenovo, and Supermicro
• AMD EPYC CPUs can achieve higher throughput and faster inferencing than competitors
• AMD EPYC CPUs are compatible with x86 architecture software and many examples of open source AI software, such as those from Hugging Face and OpenAI

When a business is looking to build private AI infrastructure, choosing the right CPU for its small and medium AI inference models is a key part of the decision-making process.

One choice is AMD EPYC CPUs, which are part of the AMD end-to-end portfolio of AI solutions.

These top tips will help you decide if AMD EPYC CPUs are the right CPUs for your AI models.

Choose an infrastructure partner whose CPUs are validated by manufacturers

When choosing the right CPU for your AI inferencing infrastructure, it’s important to consider if it will ‘play nicely’ with other hardware you might be considering. After all, while CPUs are essential for building AI infrastructure, they aren’t the only components.

AMD EPYC Server CPUs are validated for

• HPE ProLiant DL145, DL325, DL365, and DL385 server platforms
• Lenovo ThinkEdge SE455, SR635, SR645, SR655, and SR665 server platforms
• Supermicro WIO Systems
• Supermicro AS-E300-14GR, AS-1115S-FWTRT, and AS-1115-RDWTRT servers
• And more

Consider a high-performance CPU that can run AI models if you’re on a budget

While GPUs are often the key talking point of AI infrastructure discussions, a more cost-effective approach for some workloads can be to use a CPU-only system.

CPU-only systems can support models up to 13 billion parameters, according to AMD’s 2026 Deploying AI on a budget white paper. These systems can support:

• Classic machine learning tasks that use decision trees, random forests, or linear statistical forests for tasks like sentiment analysis, fraud detection, and image classification. These processes actually benefit from the linear processing of a high-core-count CPU model versus GPUs' parallel processing.
• Interactive AI agents, which can be tuned for specific tasks. These smaller models are far more efficient than larger models and so can work well using only CPUs.
• Deep learning vision models and pattern recognition computer vision models. CPUs can be more responsive than GPUs in certain scenarios, such as edge computing workloads like object detection or facial recognition.
• Memory-intensive graph analysis, where CPUs' greater memory speed and capacity outperform GPUs.
• Recommendation systems, once again, thanks to CPUs' memory capabilities and ability to support high-speed RAM.

AMD EPYC CPUs once again excel in this area. AMD EPYC server CPUs hold over 500 performance workload CPUs and outperform the competition in many areas.

For example, the 2P EPYC 9965 server CPU has up to 3.8x the throughput for end-to-end AI versus the 2P Intel Xeon 8592+ in TPCxAI @SF30, according to internal AMD testing¹.

The same AMD EPYC CPU also helps accelerate businesses’ AI journey with up to 93% faster throughput ranking and classification using XGBoost compared to the 2P Intel Xeon 6980P, 32C/FP32².

Choose a CPU that can work in harmony with high-performance GPUs

If you decide that your workloads require GPU as well as CPU infrastructure, make sure that both chipsets work optimally with each other. As AMD EPYC CPUs are optimized for AI CPU inference, they can enhance GPU accelerators’ AI efficiency.

Speaking at AMD Advancing AI 2025, CEO Lisa Su said: “We're adding the equivalent of billions of new virtual users to the global compute infrastructure. All of these agents are here to help us, and that requires lots of GPUs and lots of CPUs working together in an open ecosystem.”

AMD EPYC CPUs work in harmony with GPUs from AMD and other vendors. In testing AMD EPYC 9575F CPU-based servers with eight GPUs achieved up to 13% faster time-to-first-token and 6% higher overall inference throughput than an eight-GPU server using Intel Xeon 6960P CPUs ^{a b}.

Think about compatibility and openness

When overhauling your datacenter infrastructure, the last thing you want to do is destabilize your software stack. Choosing a CPU for your AI workloads that works with the existing data center software setup is, therefore, vital for a smooth journey to AI adoption.

AMD EPYC 9005 server CPUs are x86 software compatible, which means they can integrate easily into existing x86 infrastructure. While some software rearchitecting may be necessary, it should be minimal – yet another way in which AMD EPYC CPUs help accelerate companies’ AI journey.

It’s not just software compatibility that’s important, though. Over recent years, businesses have become increasingly aware of vendor lock-in and the problems it can create for businesses. Consequently, IT decision makers are increasingly interested in open source options; a 2025 survey from McKinsey found 50% of respondents were using open source AI technologies across several areas of their AI software stack.

AMD has alliances with many open source software providers, including Hugging Face, Lamini, and OpenAI. The company is also involved in cross-platform initiatives like MLIR, OpenMP, OpenXLA, PyTorch, TensorFlow, and Triton.

AMD has also developed its own open software stack, ROCm™, which it describes as providing “open and easy-to-use tools built around industry standards” that allow businesses to create “optimized portable software”. AMD ROCm is an open software platform, with much of the source code – including drivers, tools, and libraries – published on the well-known software repository GitHub.

Footnotes

¹ 9xx5-012:TPCxAI @SF30 Multi-Instance 32C Instance Size throughput results based on AMD internal testing as of 09/05/2024 running multiple VM instances. The aggregate end-to-end AI throughput test is derived from the TPCx-AI benchmark and as such is not comparable to published TPCx-AI results, as the end-to-end AI throughput test results do not comply with the TPCx-AI Specification. 2P AMD EPYC 9965 (384 Total Cores), 12 32C instances, NPS1, 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1DPC, 1.0 Gbps NetXtreme BCM5720 Gigabit Ethernet PCIe, 3.5 TB Samsung MZWLO3T8HCLS-00A07 NVMe®, Ubuntu® 22.04.4 LTS, 6.8.0-40-generic (tuned-adm profile throughput performance, ulimit -l 198096812, ulimit -n 1024, ulimit -s 8192), BIOS RVOT1000C (SMT=off, Determinism=Power, Turbo Boost=Enabled) versus 2P Xeon Platinum 8592+ (128 Total Cores), 4 32C instances, AMX On, 1TB 16x64GB DDR5-5600, 1DPC, 1.0 Gbps NetXtreme BCM5719 Gigabit Ethernet PCIe, 3.84 TB KIOXIA KCMYXRUG3T84 NVMe, Ubuntu 22.04.4 LTS, 6.5.0-35 generic (tuned-adm profile throughput-performance, ulimit -l 132065548, ulimit -n 1024, ulimit -s 8192), BIOS ESE122V (SMT=off, Determinism=Power, Turbo Boost = Enabled)

Results: CPU Median Turin 192C, 12 Inst 6067.531 EMR 64C, 4 Inst 1607.417. Results may vary due to factors including system configurations, software versions, and BIOS settings. TPC, TPC Benchmark and TPC-C are trademarks of the Transaction Processing Performance Council. (9xx5-012)

² 9xx5-162: XGBoost (Runs/Hour) throughput results based on AMD internal testing as of 04/08/2025. XGBoost Configurations: v1.7.2, Higgs Data Set, 32 Core Instances, FP32 2P AMD EPYC 9965 (384 Total Cores), 1.5TB 24x64GB DDR5-6400 (at 6000 MT/s), 1.0 Gbps NIC, 3.84 TB Samsung MZWLO3T8HCLS-00A07, Ubuntu® 22.04.5 LTS, Linux 5.15 kernel, BIOS RVOT1004A, (SMT=off, mitigations=on, Determinism=Power), NPS=1 versus 2P Xeon 6980P (256 Total Cores), 1.5TB 24x64GB DDR5-8800 MRDIMM, 1.0 Gbps Ethernet Controller X710 for 10GBASE-T, Micron_7450_MTFDKBG1T9TFR 2TB, Ubuntu 22.04.1 LTS Linux 6.8.0-52-generic, BIOS 1.0 (SMT=off, mitigations=on, Performance Bias)

Results: CPU Throughput Relative 2P 6980P 400 1 2P 9755 436 1.090 2P 9965 771 1.928 Results may vary due to factors including system configurations, software versions and BIOS settings. (9xx5-162)