Skip to the swatches
Infoubat

Sample No. 0222

Meta Details Open AI Networking Stack at 2026 OCP Summit, From 102.4T Switches to MetaRoCE

AI & ML John Smith

Meta outlined open networking for AI clusters, including liquid-cooled 102.4T switches, a flatter scale-out fabric and open-source MetaRoCE.

Meta Details Open AI Networking Stack at 2026 OCP Summit, From 102.4T Switches to MetaRoCE

Meta’s networking announcements at the 2026 Open Compute Project Global Summit point to a broader shift in AI infrastructure: the network is being designed as a full stack, from accelerator links inside a rack to connections between regions. The company described new switch hardware, changes to its scale-out fabric and software work with outside partners. The practical question is how far these designs can make large AI systems easier to build, operate and extend across different vendors.

The work supports Meta’s latest AI models, including Muse Wonder and Muse Spark, as well as its Prometheus cluster. Meta also made the recorded summit talks and slide decks available, offering a more detailed look at the engineering choices behind the headline systems.

Switch capacity and cooling move together

At the hardware level, Meta presented IcePack, its first liquid-cooled network rack, and IceCube, a 102.4-terabit-per-second switch designed for it. The switch uses 128 800-gigabit Ethernet connections and comes in versions built around Broadcom Tomahawk 6C or NVIDIA Spectrum-6 silicon. IceCube and N-IceCube use a 2OU design that combines liquid and air cooling and omits external physical-layer devices. Both run Meta’s FBOSS network operating system.

The design links cooling, physical layout and switch capacity in a single platform. That matters at AI scale, where network equipment adds to the power and heat demands of densely packed systems. Meta also discussed integrated optics, drawing on validation of 51.2T co-packaged optics as it considers designs for 102.4T. Its sessions on linear pluggable optics and near-package optics explored ways to manage the power, density and reliability tradeoffs of moving data between systems.

Short connections are another focus. Meta described large-scale deployment of 800G active electrical cables and the component quality work involved. These cables address links over shorter distances within data centers, where the cost and reliability of each connection can add up across a large fleet. The talks examined whether linear optics can avoid the per-module retimer DSP, which adds power, heat and cost, while shifting more equalization and troubleshooting work to the host.

A flatter fabric for scale-out

For connections among accelerators across a data center, Meta described its Flattened Planar Fabric. In this multi-plane design, each GPU connects to eight network planes. The fabric moves from a three-tier to a two-tier Clos topology, uses packet spraying at the network interface card, and combines host and switch control planes. Meta developed the approach with NVIDIA and said it is sharing it with cloud partners.

The structure is intended to support AI traffic patterns in which many accelerators exchange data during training. The summit sessions covered the operating work behind that goal, spanning GPUs, scale-up links, FBOSS, drivers, communication libraries and the scheduler. One talk on NVIDIA Spectrum-X and FBOSS focused on how the switch, software development kit, SAI implementation and simulator fit into a production fabric, including the testing and release workflow used for joint integration.

Meta also discussed using cloud GPU capacity alongside its own infrastructure. The session focused on applying lessons from on-premises deployments to cloud networking, with performance and reliability as the central concerns. The source doesn’t give comparative performance figures, so the talks are most useful as design and operations context rather than as evidence of a measured advantage.

Extending the network across sites

Meta’s scale-across work connects buildings, campuses and regions. Its Backend Aggregation system, or BAG, is a petabit-scale super-spine used in the Prometheus cluster. Meta’s summit material described the system’s architecture and deployment challenges, including fitting platforms together, managing distance, maintaining reliability and planning capacity as AI requirements change.

For regional backbones, Meta presented an SRv6 design running on FBOSS and SAI. The talk covered the production data and control planes, including the use of shallow buffers across wide-area network distances. Meta said it took the design from concept to deployment within months. A separate panel with participants from Meta, Google Cloud, Microsoft Azure, Broadcom and Arista Networks compared what routing and optical transport need to support AI clusters at this scale.

The operating model matters as much as the topology. Meta’s sessions on large-cluster reliability and BAG describe the network in the context of the full training system, where a slow component can affect distributed work across many accelerators. The summit also examined how to bridge on-premises systems and cloud GPU capacity, an increasingly relevant operational question as organizations draw on more than one source of compute.

Ethernet moves into scale-up

Meta introduced Tahan, an Ethernet scale-up switch for its MTIA 300 system, based on Broadcom Tomahawk 5. The company also previewed a rack design for 72 accelerators using 102.4T silicon. These efforts test Ethernet in scale-up networking, where accelerators within a system need fast, tightly coordinated communication.

Meta and other participants have also been developing ESUN, or Ethernet for Scale-up Networking. The 1.0 specification was released earlier in the year, and work on version 1.1 is progressing. Summit sessions covered validation methods for the ESUN ecosystem and testing copper links between accelerators. In that copper design, thousands of connections per rack make yield and serviceability significant concerns: blind-mate cartridges can leave individual links difficult to access, so Meta framed validation as a statistical problem.

Open software and platform standards

Meta, Microsoft and Alibaba are co-leading OSPA, an OCP initiative to standardize the switch platform layer. The 1.0 specification has been submitted to OCP. Meta’s case is that SAI has helped make data-plane software portable, while platform software remains fragmented and can require duplicated bring-up work for each switch design. A shared reference design could reduce that repeated effort across vendors; its value will depend on adoption and the work needed to qualify real platforms.

Meta also announced that it is open-sourcing MetaRoCE, its RDMA transport for AI networks. Distributed training operations such as all-reduce and all-to-all can be held back by the slowest participant, making transport behavior part of system performance. The initial user-space software implementation, libsoftmetaroce, is available on GitHub, with a DPDK-based version planned. Meta presented the transport specification and software at the summit alongside sessions on platform bring-up and qualification for open FBOSS systems.

The talks, videos and slide decks are available through the 2026 OCP Global Summit recorded-content page. The next useful signal will be how these specifications, software and hardware designs perform as other teams adopt them: open interfaces can reduce duplicated engineering, but they still need broad implementation and production experience to show where the benefits hold.

Source: John Smith · engineering.fb.com

Pairs well with