SemIsrael Blogs

AI is pushing data center design into new territory. As training clusters grow from dozens to hundreds, and now thousands, of accelerators, the network can no longer be treated as a background function. It has become one of the most critical factors determining overall system performance.

In these systems, accelerators are constantly exchanging intermediate data and synchronizing at every step. It’s not just about moving data fast—it’s about moving it predictably. Even small delays can ripple across the system, leaving expensive compute resources idle.

Figure 1. AI Accelerator Clusters Need Tightly Orchestrated Interconnects to Scale Efficiently

The AI Networking Challenge: It Only Takes One Slow Packet

Traditional Ethernet was designed as a best-effort network — highly efficient at moving large volumes of data, but built on the assumption that occasional packet loss, retries, and variable latency are acceptable. This model works well for general-purpose cloud workloads, but breaks down for AI systems.

AI workloads are constrained by tail latency — the slowest packet in the system. Even if most data arrives quickly, a single delayed packet can stall the entire cluster, leaving expensive accelerators idle. Packet loss makes this worse: retransmissions introduce additional delay and variability, further impacting synchronization across the cluster. AI networks require lossless or near-lossless behavior and highly deterministic latency — far beyond what traditional Ethernet was designed to provide.

These inefficiencies compound at scale. Bandwidth waste and excessive retransmissions translate directly into higher power consumption, increased infrastructure cost, and reduced scalability. As clusters grow to hundreds of thousands of accelerators, this becomes unsustainable.

From a system perspective, the goal of a scale-up network is to make the network effectively disappear. Accelerators should behave as if they are part of a single, unified machine, with communication that feels local rather than remote. Achieving this requires a network that is not only fast, but also predictable, efficient, and tightly integrated with the compute fabric.

To continue reading the full blog – click here