Open infrastructure designs for scaling modern artificial intelligence networks
Recent industry disclosures highlight open networking hardware and software architectures designed to support large-scale artificial intelligence clusters.
2 min read·Tuesday, 27 October
Building artificial intelligence systems requires rethinking how data moves across clusters of specialized computer chips. Modern training workloads demand massive interconnect bandwidth to keep thousands of processors running smoothly without idling.
New infrastructure designs incorporate advanced thermal management to handle dense switch equipment. IcePack and IceCube represent liquid-cooled network racks featuring 102.4T switch blades. These incorporate either Broadcom Tomahawk 6C or NVIDIA Spectrum-6 application-specific integrated circuits, providing 128 ports of 800-gigabit ethernet with non-blocking capacity.
Scale-up and scale-out fabrics
Connecting accelerators within a single rack and across entire data centers requires different strategies. Ethernet scale-up uses high-speed connections to treat a rack of chips as a single unified system. Tahan is an ethernet-based scale-up switch built for rack-scale accelerator systems, using Broadcom Tomahawk5 silicon to support up to 72 accelerators per rack.
For larger deployments, flattened planar fabrics connect each graphics processing unit across multiple network planes using a two-tier structure. Meanwhile, backend aggregation networks link multiple buildings, campuses, and regions into massive gigawatt-scale clusters.
Open standards and software
Proprietary switch platform layers often duplicate bring-up work across different hardware vendors. To address this, an initiative called the Open Switch Platform Architecture introduces a unified reference design, with its version 1.0 specification submitted to the Open Compute Project. Additionally, remote direct memory access transports — which allow computers to transfer data directly memory-to-memory without involving the operating system — are moving to open source with implementations such as MetaRoCE.
Google AI Edge Gallery & Gemma 4 Setup GuideFree guide
Comments