SemIsrael Blogs

CPUs are having a moment: Inside Arm’s hyperscale surge

Microsoft and AWS earnings just made the case: Arm-based CPUs are now core infrastructure for agentic AI.

Two earnings calls, one story. In the span of 24 hours, Microsoft and Amazon each delivered proof that Arm-based custom silicon has moved from experiment to backbone of hyperscale cloud infrastructure.

Microsoft CEO Satya Nadella put the architectural shift in nine words: “When it comes to running agents, CPUs are just as important as GPUs.” Then he backed it up: By the end of July, Microsoft expects Cobalt 200 racks running in more than 25 data centers worldwide.

A day later, Amazon reported its chips business (Graviton, Trainium and Nitro combined) had blown past a $25 billion annual revenue run rate, growing triple digits year over year. Graviton revenue commitments nearly tripled quarter over quarter, and Graviton5 is being adopted almost twice as fast as Graviton4 did.

Different products, different clouds, same direction: the biggest hyperscalers are building Arm-based CPUs as core AI infrastructure, tuned to their own workloads and their own economics.

Agents need more than model compute

Generative AI’s first wave was measured in model size, accelerator supply, tokens out. Agentic AI is more complex and messier. Agents retrieve data, call tools, run code, apply policies, check their own work. That’s a wider job.

GPUs still perform the bulk of model training and inference token generation, but CPUs run everything around them: routing requests, preparing data, orchestrating tools, managing memory and storage, keeping thousands of concurrent interactions moving without falling over.

For Microsoft, its Cobalt VM portfolio already carries first-party Microsoft services plus customer workloads from Adobe, Arm, Elastic, OpenAI, Sprinklr and TomTom, Nadella said during their earnings call. That’s a real production footprint.

AWS is telling the same story from a different angle. Graviton5is built for agentic AI workloads such as real-time reasoning, code generation, and multi-step orchestration. Graviton5 powered M9g instances deliver up to 25% better compute performance than Graviton4-based M8g instances and up to 35% faster machine learning inference.

To continue reading the full blog – click here

MORE >

GOLD PARTNERS