You have already used regular expressions once in this series when we told deepdiff to ignore uptime fields. I promised we would cover them properly later, and now is the time.
Regular expressions (regex for short) are a way to describe patterns in text. Instead of asking "does this string contain the exact word up?" you can ask "does this string contain any four-digit number followed by a slash and another number?" That kind of question is hard to answer with the string methods we have used so far, but regex handles it easily.
For a network engineer, regex comes up in a bunch of places. Extracting an IP address from a config line. Pulling interface names out of a log message. Parsing show output when there is no ready-made TextFSM template. Validating that a value looks like a MAC address before you use it.
Regex has a reputation for being cryptic, and honestly, it is cryptic. A complex regex pattern can look like someone leaned on the keyboard. But you do not need to master all of it to be useful. Learning even the basics unlocks a lot of what you will actually need day to day. In this part, we will cover the parts that come up most often, and I will keep the examples focused on things a network engineer will actually parse.
Why Regex
Let's start with a simple problem to see where string methods doesn’t always work well. Say you have a line of text from a device and you want to pull out the interface name.
Interface Ethernet0/1 is up, line protocol is up
With string methods, you could do something like this.
line = "Interface Ethernet0/1 is up, line protocol is up"
interface = line.split()[1]
print(interface)
Output
Ethernet0/1
That works. But what if the line format changes slightly, like an extra word at the start, or the interface appears in a different position? Your code breaks. What if you have a whole file with many different lines and you only want to find the ones that contain an interface name at all? Now you are stacking .split(), if in, and slicing to try and cover every case. It gets messy fast.
Regex describes what an interface name looks like, rather than where it happens to sit in the string. Something like "a word starting with Ethernet, followed by a number, a slash, and another number." Once you have that pattern, you can find interface names anywhere in any text, without caring about the surrounding words or the exact format of the line.
The re Module
Python's regex support lives in the built-in re module. You do not need to install anything, just import it.
import re
The main functions you will use are:
re.search()- looks for the first match anywhere in the stringre.match()- only matches at the beginning of the stringre.findall()- returns all matches as a listre.sub()- substitutes matches with something elsere.compile()- pre-builds a pattern you can reuse (we will cover this later in the series)
For most day-to-day work, re.search() and re.findall() should be enough.
Here is the simplest possible regex example, looking for the word up in a string.
import re
line = "Interface Ethernet0/1 is up, line protocol is up"
result = re.search("up", line)
print(result)
Output
<re.Match object; span=(21, 23), match='up'>
re.search() returns a match object if it finds a match, or None if it does not. The match object holds information about where the match happened and what was matched. You do not usually print it directly, you either check if it exists (which tells you whether there was a match) or call methods on it to get the actual matched text.
import re
line = "Interface Ethernet0/1 is up, line protocol is up"
result = re.search("up", line)
if result:
print("Match found")
print(f"Matched text: {result.group()}")
print(f"Position: {result.start()} to {result.end()}")
Output
Match found
Matched text: up
Position: 25 to 27
result.group() gives you the actual text that matched, result.start() and result.end() give you the position in the string. In this simple example the pattern is just a literal string, so the match is not that interesting yet. Where regex starts to shine is when the pattern describes a shape rather than an exact string, which is what we will look at next.
Basic Patterns
So far, we have only matched literal text like "up". That is not much better than using the in keyword. The real use of regex comes from metacharacters, special characters that let you describe patterns instead of exact strings. Let's go through the ones you will use most often.
Character Classes
Character classes match a category of characters instead of one specific character. The most common ones are listed below.
\d- any digit (0-9)\D- any non-digit\w- any word character (letters, digits, or underscore)\W- any non-word character\s- any whitespace (space, tab, newline)\S- any non-whitespace.- any character except a newline
Let's use these to find a number in a string.
import re
line = "Total ports: 48"
result = re.search(r"\d", line)
print(result.group())
Output
4
The pattern \d matches a single digit. re.search() returns the first match, so we got 4, the first digit in the string. That is not very useful yet, we probably want the whole number, not just the first digit. That is where quantifiers can help.
Also notice the r"..." around the pattern. That is a raw string, and it stops Python from treating backslashes as escape characters. Regex uses backslashes a lot, so always use raw strings for your patterns. If you forget the r, most patterns will still work, but you can run into confusing bugs.
Quantifiers
Quantifiers tell regex how many times to match the previous character or class. The main ones are:
*- zero or more times+- one or more times?- zero or one time (optional){n}- exactly n times{n,m}- between n and m times
Let's fix our number-matching example.
import re
line = "Total ports: 48"
result = re.search(r"\d+", line)
print(result.group())
Output
48
Now the pattern is \d+, which means "one or more digits." Regex is greedy by default, meaning it grabs as many digits as it can, so 48 is matched as a single number instead of 4 and 8 separately.
Here is another example. Say you want to match a version number that might have two or three parts, like 17.12 or 17.12.1.
import re
for line in ["Version 17.12", "Version 17.12.1"]:
result = re.search(r"\d+\.\d+(\.\d+)?", line)
print(result.group())
Output
17.12
17.12.1
The pattern reads as, "one or more digits, a dot, one or more digits, and optionally a dot and more digits." The (\.\d+)? at the end is a group with a ?, which makes the whole thing optional. We will look at groups properly in the next section.
\d+ \. \d+ (\.\d+)?
| | | |
| | | └── optional group: a dot followed by more digits
| | | (matches ".1" if present, otherwise skipped)
| | |
| | └── one or more digits (matches "12")
| |
| └── a literal dot (matches ".")
|
└── one or more digits (matches "17")
Notice that the dot in the pattern is escaped as \.. That is because a plain dot in regex means "any character." If we did not escape it, 17X12 would also match. Escaping it with a backslash tells regex we want a literal dot.
Anchors
Anchors are special characters that match positions rather than actual text.
^- the start of the string$- the end of the string
Anchors are useful when you want to make sure the pattern is at a specific position, not just somewhere in the string.
import re
for line in ["r1_lnd_dc", "backup_r1"]:
if re.search(r"^r1", line):
print(f"{line} starts with r1")
else:
print(f"{line} does not start with r1")
Output
r1_lnd_dc starts with r1
backup_r1 does not start with r1
Without the ^, both lines would match, because both contain r1 somewhere. With the anchor, only the ones that actually start with r1 are matched.
$ works the same way but for the end of the string. Together, ^ and $ are how you say "the whole string must match this pattern, nothing before, nothing after."
Character Sets
You can also define your own character class using square brackets. This is called a character set.
[abc]- matches a, b, or c[a-z]- matches any lowercase letter[A-Z]- matches any uppercase letter[0-9]- matches any digit (same as\d)[^abc]- matches anything except a, b, or c (the^inside brackets means "not")
Character sets are useful when the built-in classes are not specific enough. For example, matching a valid hex character.
import re
result = re.findall(r"[0-9a-fA-F]+", "MAC address: 00:1A:2B:C3:D4:E5")
print(result)
Output
['AC', 'add', 'e', '00', '1A', '2B', 'C3', 'D4', 'E5']
The pattern [0-9a-fA-F]+ matches one or more characters that are digits or letters a-f (either case). Notice the output picked up more than just the MAC address. AC, add, and e all came from the words MAC address because those letters happen to be valid hex characters too. This is a common regex gotcha, your pattern matches exactly what you asked for, and sometimes that includes text you did not mean to include.
A cleaner pattern would include the colons and match the full MAC address as one piece. That is something we will come back to once we have covered groups and can start writing more targeted patterns.
Groups and Capturing
So far, we have been matching whole patterns and getting the whole match back. But often, you do not just want to know that a pattern matched, you want to pull specific pieces out of it. This is where groups come in.
You create a group by wrapping part of your pattern in parentheses. Anything inside the parentheses becomes a group that you can access separately from the rest of the match.
Let's start with a simple example. Say you want to match an interface line and pull out just the interface name and the status.
import re
line = "Interface Ethernet0/1 is up, line protocol is up"
result = re.search(r"Interface (\S+) is ([a-zA-Z]+)", line)
print(result.group())
print(result.group(1))
print(result.group(2))
Output
Interface Ethernet0/1 is up
Ethernet0/1
up
The pattern has two groups. (\S+) matches one or more non-whitespace characters, which is a common way to grab a "word" out of text. ([a-zA-Z]+) matches one or more letters and nothing else, so it stops before the comma that follows up. Without that stricter pattern, \S+ would happily include the comma too and return up,.
result.group() (or result.group(0)) returns the entire matched string. result.group(1) returns the first group, result.group(2) returns the second, and so on. This is how you extract specific pieces out of a larger match.
You can also unpack all the groups at once with .groups().
import re
line = "Interface Ethernet0/1 is up, line protocol is up"
result = re.search(r"Interface (\S+) is ([a-zA-Z]+)", line)
interface, status = result.groups()
print(interface)
print(status)
Output
Ethernet0/1
up
.groups() returns a tuple of all the groups, in order. Then you can unpack it into named variables in one line, which reads much better than calling .group(1), .group(2), and so on.
A Better MAC Address Example
Remember the messy MAC address example from earlier? Let's clean it up now that we have groups.
import re
line = "MAC address: 00:1A:2B:C3:D4:E5"
result = re.search(r"([0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5})", line)
if result:
print(result.group())
Output
00:1A:2B:C3:D4:E5
The pattern reads as, "two hex characters, followed by five repetitions of a colon and two more hex characters." That is a stricter description of what a MAC address looks like, and it only matches the full thing.
The (?::[0-9a-fA-F]{2}){5} bit uses a non-capturing group, marked with (?:...). Non-capturing groups let you group things for the sake of the pattern (so you can apply a quantifier like {5} to the whole thing) without adding an extra group to the result. If we had used a regular (...) group, we would end up with an unwanted extra entry when we called .groups(). Use non-capturing groups whenever you need to group without capturing.
To make this a bit more visual, let's break down why used a non-capturing group. We want to match this MAC address structure.
00 : 1A : 2B : C3 : D4 : E5
│ └────────────────────┘
│ 5 repeats of ":XX"
│
└─ the first pair, matched separately
The first pair (00) is matched by itself. Everything after it is a pattern that repeats five times, colon + two hex chars.
To repeat a pattern, regex needs to know what "the pattern" is. Without a group, quantifiers only apply to one character or class at a time.
Without a group (broken)
[0-9a-fA-F]{2}:[0-9a-fA-F]{2}{5}
│
└── {5} applies only to {2}, breaks the pattern
The {5} at the end only affects the very last thing, which is {2}. Regex does not know we meant "repeat the whole colon+hex thing." So this fails.
With a normal capturing group (works, but noisy)
[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}
└──────────────┘
This whole thing is one unit,
repeated 5 times by {5}
Wrapping it in (...) bundles :XX together into one unit, and {5} applies to the whole unit. But this is a capturing group. That means:
result.group() >> "00:1A:2B:C3:D4:E5" (whole match, always available)
result.group(1) >> ":E5" (last repeat only, saved as side effect)
We do not care about :E5 on its own. It is just noise from the fact that we grouped for structural reasons, not because we wanted to extract anything.
With a non-capturing group (best of both)
[0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5}
└───────────────────┘
Still one unit, repeated 5 times,
but not saved separately
The ?: at the start of the group tells regex, "group this so I can apply {5} to it, but do not save it as a captured group."
result.group() >> "00:1A:2B:C3:D4:E5"
result.groups() >> () empty, nothing captured
There is more than one way to write a regex that matches a MAC address. I picked this pattern because it gives me a good excuse to show non-capturing groups.
findall With Groups
re.findall() behaves differently when your pattern has groups. Without groups, it returns a list of matched strings. With groups, it returns a list of tuples, one tuple per match, containing just the captured groups.
Say you have a config file with multiple interfaces and you want to pull them all out.
import re
config = """
interface Ethernet0/1
description LINK TO CORE
interface Ethernet0/2
description LINK TO EDGE
interface Loopback0
description ROUTER ID
"""
matches = re.findall(r"interface (\S+)\s+description (.+)", config)
for interface, description in matches:
print(f"{interface}: {description}")
Output
Ethernet0/1: LINK TO CORE
Ethernet0/2: LINK TO EDGE
Loopback0: ROUTER ID
Let's break the pattern down piece by piece.
interface- the literal wordinterfacefollowed by a space. Regex matches this exactly as it is.(\S+)- the first capturing group.\S+matches one or more non-whitespace characters, which is how we grab the interface name likeEthernet0/1orLoopback0.\s+- one or more whitespace characters. This covers the newline between theinterfaceline and thedescriptionline, plus the leading space in front ofdescription.description- the literal worddescriptionfollowed by a space.(.+)- the second capturing group..matches any character except a newline, and+means one or more, so this grabs the whole rest of the description text until the end of the line.
So the pattern reads as, "the word interface followed by an interface name, then some whitespace, then the word description followed by the description text."
Two groups in the pattern, so findall returns a list of tuples with two elements each. The for interface, description in matches: loop unpacks each tuple into two variables, and we print them out.
