SIAgentNews.com
← All news
ConfirmedAgentsChips

Lambda adopts NVIDIA Photonics to power large-scale AI factories

The cloud provider is using co-packaged optics to connect over 10,000 GPUs with significantly lower latency and better power efficiency.

· First reported by blogs.nvidia.com

Blueprint drawing: picture for the story 'Lambda adopts NVIDIA Photonics to power large-scale AI factories'
AI-made blueprint drawing for this story. No real people or products are shown.
5ximprovement in power efficiency

The short version

  • Lambda is connecting more than 10,000 GPUs using NVIDIA Photonics co-packaged optics (CPO) switches.
  • The new networking setup reduces latency by about 4% at the rack scale and up to 10% at the pod scale.
  • The technology offers up to 5x better power efficiency and up to 15x higher reliability than standard pluggable optics.
  • Technicians can replace individual light sources in the field instead of replacing entire optical modules.
  • The system uses liquid cooling to maintain a median optics temperature of 36°C.

Why it matters

As AI models grow larger, the network connecting the GPUs becomes a bottleneck for both speed and electricity costs. This hardware shift helps providers run massive training jobs with fewer interruptions and less wasted power.

Scaling the AI factory

Training modern AI models requires thousands of GPUs to work in perfect synchronization. To achieve this, cloud providers need a network that can move data quickly while remaining stable enough to handle massive workloads without crashing. Lambda is addressing these requirements by integrating NVIDIA Photonics co-packaged optics (CPO) switches into its infrastructure.

The company is using these switches to connect NVIDIA GB300 NVL72 racks, which house more than 10,000 GPUs. By focusing on the networking layer, Lambda aims to create a more resilient environment for high-demand computing. This infrastructure is designed to support the next generation of AI by ensuring that the hardware can keep pace with the software's demands.

Dave Ward, president of cloud services at Lambda, noted that networking must keep up with the demands of increasingly large AI clusters. He described bringing co-packaged optics into a deployment of this scale as an important step for the industry. The goal is to build a foundation that can handle the sheer volume of data required for modern machine learning.

Solving the latency puzzle

In a large training job, a single broken link in the network can cause thousands of GPUs to stall. This creates a significant productivity hurdle for companies trying to train models at scale. Lambda's tests show that NVIDIA Photonics can significantly reduce network latency, which is the time it takes for data to move between different parts of the cluster.

According to reports from blogs.nvidia.com, the switches reduced latency by about 4% at the rack scale and by as much as 10% at the pod scale. Despite these improvements in speed, the network maintains the same throughput as traditional pluggable optics. This means the system moves the same amount of data but does so more quickly and with fewer delays.

Reducing latency is particularly important for agentic inference, where one network disruption can cause cascading delays across many agents. Every moment of downtime costs time, and every watt used by the network is a watt that cannot be used for actual computation. By optimizing the network, Lambda ensures that more resources are dedicated to processing rather than just moving data.

How co-packaged optics work

Traditional networking often uses pluggable transceivers, which are separate components that plug into a switch. NVIDIA Photonics replaces these with co-packaged optical engines that sit on the same package as the switch chip. This design brings the electrical-to-optical conversion much closer to the chip, shortening the electrical path from inches to millimeters.

By shortening this path, the system reduces signal loss and removes the need for retimer chips. Retimer chips are components used to boost signals, and removing them cuts both the power draw and the total number of parts in the signal path. This streamlined architecture contributes to the 5x better power efficiency reported by the company.

The system also uses 4x fewer lasers than conventional setups, which reduces the number of components that can fail. Each switch features 18 removable external light sources that supply light to the optical engines. Because these light sources are external, technicians can replace them in the field without having to swap out the entire optical module.

Heat and reliability gains

Heat is a major enemy of high-performance computing, and managing it is essential for maintaining stable signals. Lambda's deployment uses liquid cooling to carry heat away from the switch chip and the optical engines. This cooling method helps maintain a median optics temperature of 36°C, which is significantly cooler than the 51°C seen in conventional pluggable optics.

Lower temperatures lead to more stable signal integrity, which directly impacts the reliability of the entire cluster. The move to co-packaged optics provides up to 15x higher reliability than traditional pluggable optics. This reliability is measured by the mean time between interruptions (MTBI), a metric that tracks how long a system can run before a failure occurs.

For cloud providers, higher reliability means fewer interruptions for their customers. When a network is more stable, it can support longer and more complex training runs. By combining cooler hardware with a more reliable design, Lambda is creating a more predictable environment for the massive GPU clusters required by today's AI developers.

What to watch

  • How other cloud providers will adopt co-packaged optics for their own GPU clusters.
  • Whether the 15x reliability increase significantly lowers the cost of training large models.
  • The impact of liquid cooling on the overall energy footprint of AI data centers.

Sources and links

First reported by blogs.nvidia.com. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news