Netflix Uses AMD Epyc CPUs and Mellanox Adapters to Boost Bandwidth to 400 Gbit/s

At the EuroBSD conference, Netflix has shared the configuration of the systems used for streaming content worldwide. The content provider has been able to push the throughput to 400 Gbit/s using a mix of AMD Epyc 7502p processors (32 cores) paired with 256 GB of DDR4-3200 memory. The configuration features two Mellanox ConnectX-s adapters. These are usually limited to four 100 GbR connections.

This means that the overall system throughput remains under 240 Gbit/s. However, developer Drew Gallatin shared in a lecture that he was able to optimize them to exceed data rates of 400 Gbit/s. That’s a considerable 100Gbit+ more than the usually expected 200-something.

Gallatin managed to do this by having the data transport run directly through the network hardware via a TLS offload. The FreeBSD kernel which was was used as the OS for the systems offers this possibility, therefore freeing up CPU resources. The actual TLS session is established in the userspace, but the keys used then eventually migrate using the kernel to the network hardware, where the encryption takes place. A detour via the processor is no longer necessary.

The team has not yet been able to test TLS offloading with Intel hardware. In addition, the team already has hardware prototypes that could enable network connections up to 800 Gbit/s, but these have not yet been tested either.

Source: FreeBSD (Via: Hardware.nl)

Areej Syed

Processors, PC gaming, and the past. I have written about computer hardware for over seven years with over 5000 published articles. I started during engineering college and haven't stopped since. On the side, I play RPGs like Baldur's Gate, Dragon Age, Mass Effect, Divinity, and Fallout. Contact: areejs12@hardwaretimes.com.
Back to top button