Using Python and JNJA2 for IOS Config Templates

With all of the articles on SDN and Network Programability, I’m often asked two common questions: “Do I really need to learn Python (or some other programming language) for my job?” and “…But seriously, what can I really do with Python as a network engineer?” As is often the case in IT, I think the first question is “it depends,” for example you wouldn’t write a script to go out and collect configurations from network devices if you already had a tool that did just that and in the way you needed. On the other hand if you needed that tool to also scrape the configurations for vulnerabilities or a set of best practices you may need to look at a new tool or think about making some customization yourself.

For this post I wanted to take a look at the latter question and share a project I recently created to help a customer with deploying a large handful of edge routers that had a very repeatable configuration template.

SDN-notepad.jpg

The project involved taking a large CSV file representing 100s of Hub and Spoke routers and generating the relevant configuration files for each.

csv

As a simple google search can tell you, there are countless off the shelf and home grown solutions to this problem, but what I wanted was something that could be easily adapted to different templates(Spoke vs Hub or ISR4K vs ISRG2, etc) and as a bonus something that I could extend later so that the configuration files could be used by a provisioning server like APIC-EM Plug n Play. I especially did not want the input file to be dependent on the order the variables were entered, but rather rely on the column headings.

After all I didn’t want to put in the time and effort for something I could only use once. For this I decided to focus on leveraging Python and Jinja2. You can find the relevant files on my Github at Template Project

Jinja2 Crash Course

Jinja2 is a popular text based template language that was developed specifically for Python and thus uses many familiar Python expressions and has a very easy to use library within Python. Many of the Jinja2 template examples are based on generating HTML pages, but the template can be designed for any application. At its core it simply generates text output and inserts the appropriate text wherever variables are defined using double curly brackets {{myVar}}.

router bgp {{BGP_AS}}
 bgp router-id {{BGP_ID}}

The above is just the surface of what is possible, but on its own we can get very far. Jinja2 also has the ability for using traditional looping structures, conditional statements, and even template inheritance which can be great if you decide to build out a  library of Jinja2 templates for your network devices. While I didn’t find the need to use any expressions they are easily introduced using {% ..%}.

{% for interface in WAN-Interfaces %}
    interface {{ interface }}
        no switchport
{% endfor %}

This allows us to quickly iterate through a list of WAN interfaces and configure them appropriately, each iteration substitutes the variable with the name of the interface, we may have a similar loop to configure our access ports with all of the typical LAN side commands.

That’s not bad, but I don’t know Python!

This is where the beauty of Jinja2 comes in. The fact that it was written for Python means that you can very quickly begin using the templates you create, even with little Programming experience. For example the code I wrote has less than 50 lines and leverages several libraries to do all the heavy lifting like reading through the CSV file and substituting the template variables.

You can check out the sourcecode files on github, but I’ll cover the high level flow and interesting bits below.

The project contains 3 main files:

  1. pnp_config.py: which is used to set environment variables like filenames for our template and csv files. If you plan to use this project, make sure you edit this to represent your filenames.
  2. buildconfigurations.py: The meat of the project that contains the python code for loading the CSV input and generating the configuration files based on our templates.
  3. config_templateIWAN-Spoke.jnj: Our Jinja2 template file that describes the configuration for a DMVPN spoke.

Within buildconfigurations.py we leverage the python CSV library  to read through our CSV file and create a list of dictionaries, each dictionary represents a CSV row, containing the Device and its relevant configuration variables from the CSV. By using dictionaries we can take out the importance of how the CSV file is ordered and instead rely on Key:Value pairs AKA use our column headers as variable names.

csv

After running the above CSV through our function, csv_dict_list, it becomes the following List of Dictionaries, making it very easy to grab the key:value pairs when we need them.

[   {  'Role': 'Spoke',
        'hostName': 'Spoke11',
        'serialNumber': '12345678901',
        'platformId': 'ISR4351',
        'site': 'Fairfax',
        'tun101ip': '10.10.8.11'},
         ....
    {  'hostName': 'Spoke21',
        'platformId': 'ISR4331',
        'serialNumber': '22345678901',
        'site': 'Sydney',
        'tun101ip': '10.10.8.21',
        'tun101mask': '255.255.248.0',
        'tun101nbma': '65.65.65.1',
        'tun101nhs': '10.10.8.1',
        'tun102ip': '10.10.16.21'},
        ..
        ...}]

The csv_dict_list function also contains some conditional logic that only reads in rows that contain the correct role or value. For example if we had different templates and variables for Hubs vs Spokes we could use this to make sure the appropriate configurations are used when creating Spokes.

def csv_dict_list(devices, role):
    reader = csv.DictReader(open(devices, 'rb'))
    dict_list = []
    for line in reader:
        if role in line.itervalues():
            dict_list.append(line)
    return dict_list

Next, we throw it all together and use the jinja library to generate configurations for each dictionary (AKA Device) in our list and name the configuration file based on the hostname of each device.

def build_templates(template_file, devices, role):
#Create a List of Dictionaries from the CSV file that contain the keyword or role.
    dictList = csv_dict_list(devices, role)
    #setup template environment paths and directories
    templateLoader = jinja2.FileSystemLoader( searchpath="." )
    env = jinja2.Environment( loader=templateLoader )
    #Add logic here if we want to use different templates based on models
    template = env.get_template(template_file)

#Generate Configuration for each Dictionary in the List.
    for dictionary in dictList:
        config = template.render(dictionary)
        config_filename = dictionary['hostName'] + '-config'
        with open(config_filename, 'w') as config_file:
            config_file.write(config)
            print("wrote file: %s" % config_filename)

#Consider changing spoke argument to pull from pnp_config.py
if __name__ == "__main__":
    build_templates(TEMPLATE, DEVICES, 'Spoke')

There you have it, if you want to get started the source files are available on my github. You can modify the Jinja2 template to match your configuration file needs and then either create a csv or use one you already own. As long as the column headers in your CSV file match your jinja2 template variables everything should work as expected.

Summary

This idea could easily be extended to meet other needs before and after the config generation. For those of you looking at automated deployment and provisioning options take a look at Adam Radford’s blog, who takes this a step further and has a project that automates the uploading of generated config files and creates tasks to deploy them within APIC-EM PlugnPlay.

Related Links and Resources

My Github for this project
Learn Python the Hardway – Free online book with labs that is a great place to start learning the basics and advanced concepts of Python.
PacketPushers blog on Jinja2