^ Top

NANOG Meeting Presentation Abstract

Network Automation with State Machines
Meeting: NANOG68
Date / Time: 2016-10-19 12:30pm - 12:55pm
This item is webcast
Room: Regency Ballroom (Banquet Level)
Presenters: Speakers:

Yihua He

Yihua He is a principal network architect in Yahoo. He holds a PhD degree in computer science and has numerous publications in highly cited computer network conferences and journals. In the past four years, he has focused on researching, designing, prototyping and implementing Yahoo's next generation data center networks. In this presentation, he would like to share the experience of a state-machine based configuration management system that automates daily operations of a data center network end-to-end.

Zoe Blevins

Zoe Blevins is an automation guru in the network team at Yahoo. Over the last 7 years, she has designed, implemented, and operated software systems to provision, manage, monitor and audit Yahoo’s global network. She is a CCNP holder, proficient in numerous programming languages, and a multi-time hackovation winner. Her primary focus over the last year has been designing and implementing a network management system that removes the need for operators to interact directly with network gear for day to day operations, and she is eager to share her findings with the rest of the industry.
Abstract: Automation has become vital to building large scale networks. However, building these networks and managing their entire life cycle with minimal human intervention remains a challenge. We realize that the fundamental action in this automation can be abstracted as reconciling the difference between the actual state and the desired state of the system. Guided by state machines, we have implemented a fully automated system to provision, turn-up, and manage our data center networks at Yahoo.

In this system, the network architecture is modeled as a set of configuration templates. A no touch configuration generating engine is built on top of the model. The actual state of devices within the system is collected in real time by agents running on the devices. Additional data is pulled in from external sources, such as inventory databases, to feed the templating engine. Changes to the desired state come from input by engineers via the API, as well as state data collected from the devices. These changes then trigger the model to generate desired configurations for devices.

Once a new version of a configuration has been generated, it advances through 3 states, GENERATED, RELEASED, and VALIDATED. These states are used to track the progress of a change and control the rate and sequence at which new configurations are released out into the network. The transition from GENERATED to RELEASED is where the rate and sequence of such changes are controlled, and will be explored in depth as part of this presentation. Once a configuration is in the RELEASED state, it is ready to be picked up by the network device. The device will then apply the configuration, run a series of health checks, and report the version of the active configuration to the system.

This presentation will cover the overall design of the system, share the details of the state machines, walk through a specific use case, and discuss challenges faced when implementing the system.
Files: pdfNetwork Automation with State Machines(PDF)
youtubeNetwork Automation with State Machines
Sponsors: None.

Back to NANOG68 agenda.

NANOG68 Abstracts

  • Conference Opening
    Speakers:
    David Temkin, Netflix; L Sean Kennedy, XO Communications; Josh Snowhorn, CyrusOne;
  • Conference Opening
    Speakers:
    David Temkin, Netflix; L Sean Kennedy, XO Communications; Josh Snowhorn, CyrusOne;
  • Conference Opening
    Speakers:
    David Temkin, Netflix; L Sean Kennedy, XO Communications; Josh Snowhorn, CyrusOne;
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .
  • Security Track
    Moderators:
    Jesse Sowell, Standford University; Panelists:
    k claffy, CAIDA; Nolan Berry.
    Cory Schwartz.
    Speakers:
    John Kristoff, DePaul University; Yiming Gong.
    Qiang KeQihoo 360; .

 

^ Back to Top