Nvidia, Arm unveil server CPU plans in AI hardware shift

Sarah Mills | GlobalBeat

Nvidia and Arm revealed server-class central processing units (CPUs) designed for artificial intelligence workloads at the Computex trade show in Taiwan on Monday.

The announcements mark a pivot back to general-purpose processors after years of graphics chips dominating AI acceleration.

CPUs had been relegated to control-plane tasks while GPUs handled matrix math. Cloud operators told analysts that inference costs now eclipse training, forcing a hardware rethink.

Nvidia chief executive Jensen Huang said the Grace Blackwell Superchip pairs 72 Arm Neoverse cores with two Blackwell GPUs on one board. The module delivers 40 terabytes of coherent memory and 3.6 terabytes per second of bandwidth, according to the company. Dell, HP Enterprise, Supermicro and Lenovo plan servers this year. AWS will deploy the chips in EC2 instances starting in 2025, Huang said.

Arm chief executive Rene Haas countered with the Cortex-X925, a blueprint licensees can tweak for data-center CPUs. MediaTek, Ampere, Microsoft and Amazon already signed licenses, Arm said. The design adds a 3-nanometer version and vector extensions aimed at large-language-model serving.

Both chips aim to cut the energy used per AI query. In a live demo Nvidia claimed Grace Blackwell ran a 175-billion-parameter Llama model at one-eighth the power of current x86 racks. Haas said Cortex-based servers can “approach GPU-level throughput on small batch sizes,” citing tests by Amazon Web Services.

Wall Street investors pushed Nvidia shares up 4.2% on the news and Arm 6.1%. Shares in Intel fell 2.7%, extending a 7% slide this month. Bernstein analyst Stacy Rasgon wrote that Intel “has yet to present a competitive AI story,” in a note to clients.

Cloud providers pay roughly 3 cents per 1,000 language-model tokens on current GPU instances, UBS said in March. AWS vice-president Dave Brown told reporters that CPU-centric instances could halve that figure. “We see strong demand for inference at the edge,” Brown said during a briefing in Taipei. He declined to give pricing.

The roadmap reunites Nvidia with the CPU market it abandoned in 2011 after failed Project Denver chips. Arm has licensed server cores since 2018 but gained little share against Intel and AMD.

Market researcher IDC estimated that 92 percent of server CPUs shipped last year came from Intel or AMD. The same report said AI-related server spending topped $64 billion, up 35 percent year-over-year.

Competitors dismissed the threat. “Training large models will stay on GPUs,” Advanced Micro Devices chief executive Lisa Su told reporters on Sunday. AMD plans its own AI CPU, Turin, for 2025.

Enterprise users voiced caution. “We have code optimized for x86,” said Alex Chen, infrastructure director at Cathay Financial Holdings in Taipei. Chen said he would trial Arm servers this summer before any rollout.

Technical sessions showed Nvidia integrating Grace tightly with its CUDA software stack. Huang showed Kubernetes scheduling tasks across 256 Grace CPUs in milliseconds. Arm, by contrast, relies on open-source firmware and lets partners tune silicon.

Power consumption emerged as a key metric. FormFactor, a test-equipment vendor, measured Grace Blackwell drawing 1,200 watts at full load, 15 percent below a comparable Xeon-GPU rack. Independent engineers asked for third-party validation.

Security researchers highlighted Arm’s Confidential Compute Architecture that isolates model weights in a secure enclave. Nvidia said it will publish a white paper next month.

Shipments start in December, Huang said. Dell Technologies shares gained 3 percent on the announcement, while server cooling vendor Vertiv rose 5 percent.

Background

Nvidia exited the consumer CPU space after its Tegra line lost phone and tablet deals to Qualcomm. Arm cores, common in smartphones, have struggled to match Intel’s software ecosystem in servers.

Hyperscalers began designing custom chips in 2018 to curb x86 premiums. Amazon’s Graviton, built on Arm cores, now powers fifth-generation EC2 instances. Google and Microsoft followed with Tensor Processing Units and Maia accelerators, yet both still pair with Intel CPUs.

What’s Next

Nvidia will host a developer day in September to release firmware and benchmarking tools. Cloud providers must decide by year-end whether to lock capital budgets into Grace racks before AMD’s Turin launches in mid-2025.

Adoption may hinge on software ports. Microsoft said it will update Azure Kubernetes Service to orchestrate Arm nodes by October. Red Hat promised CentOS and RHEL builds by December.