Air Traffic Control: a network traffic conditioner

Emmanuel Bretelle, Gus Luxton, John Morrow, Andrew Pope

Audience level:
Intermediate
Category:
Other

Description

Facebook has implemented a system known as ‘Air Traffic Control’ internally that helps engineers simulate different network conditions inside Facebook’s offices. With this infrastructure, employees have the ability to control all aspects of the connection that their device has to the internet, including bandwidth, latency, packet loss, corrupted packets, and packet ordering.

Abstract

Facebook has implemented a system known as ‘Air Traffic Control’ internally that helps engineers simulate different network conditions inside Facebook’s offices. With this infrastructure, employees have the ability to control all aspects of the connection that their device has to the internet, including bandwidth, latency, packet loss, corrupted packets, and packet ordering. While 4G is gaining more presence, a lot of people are still only able to access networks with much lower connectivity quality. To be able to understand the experience those users are having while using our apps, we have created Air Traffic Control. Air Traffic Control is composed of 2 main components: ATCD, the ATC Daemon, that runs on a Linux gateway and that will apply netlink Traffic Control rules. The daemon is written in python and uses pyroute2 to set traffic control. It is able to apply traffic shaping such as: bandwidth packet loss latency packet reordering packet corruption trace network traffic for a given shaped connection for further analysis by developers An ATC frontend written in Django. The web interface has a predefined set of settings (such as different 3G, 2G scenarios and countries'average network quality experience by users) and allow users to enter their own. It communicates with the ATCD over thrift to handle requests such as start/stop shaping, start/stop packet capture. Through that interface users can access their network traces and preview their HTTP traffic in a waterfall chart.