If you have made a change on a network device, you have probably done the pre and post check dance at some point. You SSH into the device, run a few show commands, save the output somewhere, apply the change, then run the same commands again and compare the two by eye. It works, but it is slow and easy to get wrong. I have missed things this way, and I know most of you have too.
Once you get comfortable with Python, this is one of the first tasks you will want to automate. Instead of copy-pasting output into a text file and squinting at it, you can pull the data with a script, save it to a JSON file, and then run another script that tells you exactly what changed.
In this chapter, we will build a small pre and post check for BGP. I will show you how to grab BGP neighbor state from a device using NAPALM, save it to a JSON file as the pre snapshot, and then do the same again after the change to get the post snapshot. Once we have both, we will compare them with a library called deepdiff, which tells you exactly what is different between two Python data structures. We will make the BGP change outside of Python since the focus of this chapter is on the validation, not the change itself.
To keep the chapter simple, I will do this against a single device. In real life, you would run it against many devices at once, but the pattern is the same. Once you have it working for one, scaling it up is straightforward.
Our Scenario
We have five routers in total (r1 through r5), and r3 is peered with both r2 and r4. Our focus in this chapter is only on r3 and r5. We are going to make a change on r5, and we care about what r3 sees as a result. r4 is in the middle, connecting the two, but we do not touch it.
Here is r3's current view of the routes it is receiving from r4.
r3#show ip bgp neighbors 10.0.0.4 received-routes
BGP routing table information for VRF default
Router identifier 10.0.0.3, local AS number 200
Network Next Hop Metric AIGP LocPref Weight Path
* > 10.0.0.4/32 10.0.0.4 - - 100 - i
* > 10.0.0.5/32 10.0.0.4 - - 100 - 300 i
Two prefixes at the moment, 10.0.0.4/32 (from r4 itself) and 10.0.0.5/32 (from r5, propagated through r4).
The change we are planning is on r5. We will advertise a brand new prefix from r5 into BGP. Since r5 sends that prefix to r4, and r4 sends everything on to r3, we expect r3 to see an additional prefix from r4 after the change.
So this is what our pre and post check needs to catch. The pre snapshot captures the current state, two received prefixes from r4. After the change, the post snapshot should show three. When we run the diff between the two, it should highlight that the received prefix count on r4 has gone from 2 to 3. If it does not, something did not work.
We will run everything from r3's point of view, since that is the device we care about.
Capturing the Pre-Change State
The first thing we need is a script that connects to r3, pulls the current BGP neighbor state, and saves it to a JSON file. This becomes our baseline snapshot, the "before" picture that we will compare against later.
We will use NAPALM's get_bgp_neighbors() method, which returns all the BGP neighbor information as a Python dictionary. From there, it is just a case of dumping it to disk with json.dump().
import json
from napalm import get_network_driver
driver = get_network_driver("eos")
with driver(
hostname="192.168.202.103",
username="admin",
password="admin",
optional_args={"transport": "ssh"}
) as connection:
bgp_neighbors = connection.get_bgp_neighbors()
with open("r3_pre.json", "w") as file:
json.dump(bgp_neighbors, file, indent=2)
print("Saved pre-change BGP state to r3_pre.json")
Run this before making any changes on r5. It connects to r3, grabs the current BGP state, and writes it to r3_pre.json.
Here is what the file looks like after running.
{
"global": {
"peers": {
"10.0.0.2": {
"local_as": 200,
"remote_as": 200,
"remote_id": "10.0.0.2",
"is_up": true,
"is_enabled": true,
"description": "r2",
"uptime": 1508,
"address_family": {
"ipv4": {
"received_prefixes": 2,
"accepted_prefixes": 2,
"sent_prefixes": 1
},
"ipv6": {
"received_prefixes": 0,
"accepted_prefixes": 0,
"sent_prefixes": 0
}
}
},
"10.0.0.4": {
"local_as": 200,
"remote_as": 200,
"remote_id": "10.0.0.4",
"is_up": true,
"is_enabled": true,
"description": "r4",
"uptime": 1512,
"address_family": {
"ipv4": {
"received_prefixes": 2,
"accepted_prefixes": 2,
"sent_prefixes": 1
},
"ipv6": {
"received_prefixes": 0,
"accepted_prefixes": 0,
"sent_prefixes": 0
}
}
}
},
"router_id": "10.0.0.3"
}
}
Notice the received_prefixes count for peer 10.0.0.4 is 2, which matches what we saw in the show ip bgp neighbors output earlier. That is the number we expect to go up to 3 after the change.
At this point, we have our baseline. Now we make the change on r5 outside of Python.
Making the Change
The change itself is not part of what we are automating in this chapter, so we will keep it out of the script. Log into r5 and add a new prefix into BGP. It does not matter exactly which prefix, anything that gets advertised will do.
Once the change is committed on r5, give it a few seconds to propagate. r5 will advertise the new prefix to r4, and r4 will pass it along to r3. Then we are ready to capture the post-change state.
Capturing the Post-Change State
The script for the post snapshot is almost identical to the pre snapshot. The only thing that changes is the output filename.
import json
from napalm import get_network_driver
driver = get_network_driver("eos")
with driver(
hostname="192.168.202.103",
username="admin",
password="admin",
optional_args={"transport": "ssh"}
) as connection:
bgp_neighbors = connection.get_bgp_neighbors()
with open("r3_post.json", "w") as file:
json.dump(bgp_neighbors, file, indent=2)
print("Saved post-change BGP state to r3_post.json")
Run this after the change has been applied. You should now have two files in your folder, r3_pre.json and r3_post.json. Next, we will compare them.
Comparing the Snapshots
Now we have both files, we need a way to compare them. Doing this manually by opening two files side by side would defeat the whole point. We want a script that tells us what actually changed.
For this, we use a library called deepdiff. It is built specifically for comparing Python data structures like dictionaries and lists, and it gives us back a structured breakdown of the differences instead of a raw text diff. Install it with pip.
pip install deepdiff
Here is the script. It reads both JSON files, compares them, and prints the differences.
import json
from deepdiff import DeepDiff
with open("r3_pre.json") as file:
pre = json.load(file)
with open("r3_post.json") as file:
post = json.load(file)
diff = DeepDiff(pre, post, ignore_order=True)
if diff:
print(json.dumps(diff, indent=2, default=str))
else:
print("No differences found")
We load the two JSON files into Python dictionaries, pass them to DeepDiff, and print whatever comes back. If the two are identical, we print a friendly message. Otherwise we dump the diff as pretty JSON.
Run it, and you get something like this.
{
"values_changed": {
"root['global']['peers']['10.0.0.2']['uptime']": {
"new_value": 1562,
"old_value": 1508
},
"root['global']['peers']['10.0.0.4']['uptime']": {
"new_value": 1565,
"old_value": 1512
},
"root['global']['peers']['10.0.0.4']['address_family']['ipv4']['received_prefixes']": {
"new_value": 3,
"old_value": 2
},
"root['global']['peers']['10.0.0.4']['address_family']['ipv4']['accepted_prefixes']": {
"new_value": 3,
"old_value": 2
}
}
}
The output is grouped by the type of change. In this case everything falls under values_changed, meaning the field existed in both snapshots but the value is different. deepdiff gives you the full path to each change, the old value, and the new value.
Look at the last two entries. Both are on peer 10.0.0.4, and they show received_prefixes and accepted_prefixes going from 2 to 3. That is exactly what we expected. r5 advertised a new prefix, r4 passed it along to r3, and r3 now sees one more prefix from r4.
The first two entries are the uptime values for both peers, which have gone up by around 30 seconds. That is just the natural passing of time between the two snapshots and has nothing to do with our change. Which brings us to the next problem.
Ignoring Fields That Do Not Matter
The uptime values in the diff are noise. They will always be different between two snapshots because time keeps ticking, so seeing them in the output does not help us tell whether the change actually worked. In a real network, this kind of noise is even worse. Counters keep going up, timestamps keep changing, and packet counts are different every second. If your diff is full of these, the changes you actually care about get buried.
deepdiff has a way to exclude fields we do not want to compare, using the exclude_regex_paths argument. It takes a list of regular expressions, and any path in the data that matches one of them is skipped during the comparison.
Regular expressions (regex) are a topic on their own that we have not covered yet, and we will not go into them in detail here. For now, just treat the pattern below as a recipe you can adapt.
Here is the updated script.
import json
from deepdiff import DeepDiff
with open("r3_pre.json") as file:
pre = json.load(file)
with open("r3_post.json") as file:
post = json.load(file)
exclude_paths = [
r"root\['global'\]\['peers'\]\['[^']+'\]\['uptime'\]"
]
diff = DeepDiff(pre, post, ignore_order=True, exclude_regex_paths=exclude_paths)
if diff:
print(json.dumps(diff, indent=2, default=str))
else:
print("No differences found")
The one line worth understanding at a high level is the pattern itself.
root\['global'\]\['peers'\]\['[^']+'\]\['uptime'\]
Reading this from left to right, it says match any path that starts with root['global']['peers'], then has any peer IP in square brackets, and ends with ['uptime']. The [^']+ bit is what makes it match any peer IP without us having to hardcode each one. So a single pattern covers uptime for 10.0.0.2, 10.0.0.4, and any other peer that might be there.
Run the script again and the output looks a lot cleaner.
{
"values_changed": {
"root['global']['peers']['10.0.0.4']['address_family']['ipv4']['received_prefixes']": {
"new_value": 3,
"old_value": 2
},
"root['global']['peers']['10.0.0.4']['address_family']['ipv4']['accepted_prefixes']": {
"new_value": 3,
"old_value": 2
}
}
}
Now the only thing the diff shows is what we actually care about, the prefix count on 10.0.0.4 went up by one. That is a much more useful validation report.
You can add as many exclusion patterns as you like to the exclude_paths list. In a real script that compares a full device state (interfaces, routes, ARP, whatever else), you would probably build up a list of things to ignore over time as you learn what changes are just noise and what actually matters.
The Problem With Hardcoded Filenames
Right now, our capture script has the filename baked into the code. Every time we want to capture a new snapshot, we have to open the script and change r3_pre.json to r3_post.json by hand. It works, but it is annoying, and it is easy to forget. Run the script twice without changing the filename in between, and you have just overwritten your pre snapshot with the post snapshot. Not great.
The same is true for the diff script. If we want to compare a different pair of files, we have to edit the script. If we start capturing snapshots for multiple devices or over multiple days, we would need a different script for every combination, which does not scale.
What we actually want is to pass the filename in when we run the script, like this.
python capture.py --output r3_pre.json
Then before applying the change, we run it again with a different filename.
python capture.py --output r3_post.json
This is how proper command-line tools work, and it is a small change that makes our scripts a lot more reusable. Python has a built-in library for this called argparse, which is what we will use next.
Adding argparse to the Capture Script
argparse is a built-in Python library that handles command-line arguments. You describe what arguments your script takes, and argparse does the rest, parses them, provides helpful error messages if the user gets them wrong, and even generates a --help message automatically.
Here is the updated capture script.
import argparse
import json
from napalm import get_network_driver
parser = argparse.ArgumentParser(description="Capture BGP state from r3 and save it to a JSON file")
parser.add_argument("--output", required=True, help="Output JSON filename, e.g. r3_pre.json")
args = parser.parse_args()
driver = get_network_driver("eos")
with driver(
hostname="192.168.202.103",
username="admin",
password="admin",
optional_args={"transport": "ssh"}
) as connection:
bgp_neighbors = connection.get_bgp_neighbors()
with open(args.output, "w") as file:
json.dump(bgp_neighbors, file, indent=2)
print(f"Saved BGP state to {args.output}")
Three new lines at the top do all the work.
argparse.ArgumentParser(description=...) creates the parser object. The description shows up in the --help output.
parser.add_argument("--output", required=True, ...) tells the parser to expect a --output argument. Setting required=True means the script will refuse to run if this argument is missing, which is exactly what we want. The help text shows up in the --help output.
parser.parse_args() actually parses the arguments the user passed on the command line. It returns an object where each argument is available as an attribute, so args.output gives us the value the user passed.
The rest of the script is unchanged, except we use args.output instead of the hardcoded filename. Now we can run it like this before the change.
python capture.py --output r3_pre.json
And like this after the change.
python capture.py --output r3_post.json
Free Help Text
One nice thing about argparse is that it builds a --help option for you automatically. Run this and see for yourself.
python capture.py --help
Output
usage: capture.py [--help] --output OUTPUT
Capture BGP state from r3 and save it to a JSON file
options:
--help show this help message and exit
--output OUTPUT Output JSON filename, e.g. r3_pre.json
You get a proper help message just from filling in the description and help fields when defining the arguments. This becomes really handy as your scripts grow and take more options, and it makes them feel like real command-line tools instead of throwaway scripts.
If you try to run the script without the required argument, argparse catches it and tells the user what went wrong.
python capture.py
Output
usage: capture.py [--help] --output OUTPUT
capture.py: error: the following arguments are required: --output
Adding argparse to the Diff Script
The diff script has the same problem, the two filenames are hardcoded. Let's update it to take --pre and --post as arguments so we can compare any pair of snapshots.
