Co-packaged optics for HPC and data center networks

Abstract

We have witnessed a tremendous evolution in the switch industry over the past decade, which has led to a 40x increase in terms of switch Input/Output (I/O) bandwidth (BW). However, as we start reaching the physical limits of Ball/Land Grid Arrays (BGA/LGA), continued BW scaling is getting more and more challenging. A promising solution to overcome BW density and thermal cooling limits is the integration of optics onto the 1st-level package, a.k.a., copackaged optics (CO). The increased escape BW offered by CO can enable high-radix switch implementations of >150 ports, which can be combined with high data rates of ≥400 Gb/s per port. From the network design perspective, there are two key benefits of using CO: (a) the ability to build large-scale fat-tree topologies of >11,000 end points with only two switch layers, and (b) the ability to provide 4x higher bisection BW, reducing at the same time the number of required switch ASICs by an order of magnitude. CO can enable both reduced energy consumption and packet delays since fewer hops are required, i.e., packets traverse fewer SerDes lanes and visit fewer buffers, which reduces network contention and improves the tolerance to network congestion. Simulation results for synthetic traffic patterns with hotspots suggest that CO can enable linear BW scaling and can significantly reduce the mean packet delay and its standard deviation, with improvements reaching up to 71% and 79% for high-load conditions, respectively.