Python Mutable vs Immutable and References

Up to this point, we have been treating variables as labelled boxes that hold values. That mental model has worked fine for everything we have done so far, but it is actually not the full picture. As you start writing more complex scripts, especially ones that pass lists and dictionaries around between functions, you will run into some confusing behaviour that the labelled box model cannot explain.

In this chapter, we will look at what really happens when you create a variable, the difference between mutable and immutable types, and why this matters for your everyday code. We will also look at how to properly copy data when you need a truly independent version of it, including the difference between a shallow copy and a deep copy.

What Happens When You Create a Variable?

Back in Chapter 2, I described a variable as a labelled box that holds a value. That is a good starting point, but it is not quite how Python actually works. The more accurate way to think about it is as follows.

When you create a variable, Python creates the value in memory and then attaches a label to it. The variable name is the label, and the actual value lives somewhere in memory. The variable does not "contain" the value, it points to the value.

hostname = "r1_lnd_dc"

What is happening here is, Python creates the string "r1_lnd_dc" in memory, and then it attaches the label hostname to that string. If you create another variable and assign it the same value, both labels can either point to the same object in memory, or to different ones, depending on the type.

These specifics matter a lot once we start working with lists and dictionaries.

a = [1, 2, 3]
b = a

print(a)
print(b)

[1, 2, 3]
[1, 2, 3]

So far so good, both variables show the same list. But here is where it gets interesting.

a = [1, 2, 3]
b = a

a.append(4)

print(a)
print(b)

[1, 2, 3, 4]
[1, 2, 3, 4]

Wait, we only changed a, but b changed too. That is because a and b are not two separate lists, they are two labels pointing to the same list in memory. When you wrote b = a, Python did not copy the list, it just added another label to the same object.

This is the core idea you need to understand for the rest of this chapter. Some things in Python can be modified after they are created, and when you have multiple labels pointing to them, all the labels see the change.

Mutable vs Immutable Types

Some values in Python can be changed after you create them, and some cannot. The ones that can be changed are called mutable, and the ones that cannot are called immutable.

• Mutable means the value can be changed after it is created. Lists, dictionaries, and sets are mutable.
• Immutable means the value cannot be changed after it is created. Strings, integers, floats, booleans, and tuples are immutable.

When you "change" an immutable value, you are not actually changing it. You are creating a brand new value and pointing your variable at it.

x = 10
y = x

x = x + 1

print(x)
print(y)

11
10

Here, y did not change, because integers are immutable. When we wrote x = x + 1, Python created a new integer 11 and made x point to it. The original 10 is still there in memory, and y is still pointing to it.

Compare that to the list example from before. Lists are mutable, so when you modify a list, you are changing the actual object in memory, and every variable pointing to that object sees the change.

The Shared Reference Problem

Let's look at a more practical example, because this is something that bites people in real code.

Imagine you have a base set of device settings, and you want to create a few devices that share most of those settings but differ in a few details.

base_settings = ["enable_snmp", "enable_logging", "enable_ntp"]

r1_settings = base_settings
r2_settings = base_settings

r1_settings.append("enable_bgp")

print(f"r1: {r1_settings}")
print(f"r2: {r2_settings}")
print(f"base: {base_settings}")

r1: ['enable_snmp', 'enable_logging', 'enable_ntp', 'enable_bgp']
r2: ['enable_snmp', 'enable_logging', 'enable_ntp', 'enable_bgp']
base: ['enable_snmp', 'enable_logging', 'enable_ntp', 'enable_bgp']

We only meant to add "enable_bgp" to r1_settings, but it ended up on r2_settings and base_settings too. That is because all three variables point to the same list. There is only one list in memory, and three labels stuck to it.

The same thing happens with dictionaries.

default_device = {"vendor": "cisco", "model": "ISR4321"}

device_a = default_device
device_b = default_device

device_a["model"] = "ISR4451"

print(f"device_a: {device_a}")
print(f"device_b: {device_b}")
print(f"default: {default_device}")

device_a: {'vendor': 'cisco', 'model': 'ISR4451'}
device_b: {'vendor': 'cisco', 'model': 'ISR4451'}
default: {'vendor': 'cisco', 'model': 'ISR4451'}

All three are the same dictionary, just with different labels.

Passing Lists and Dictionaries to Functions

This same behaviour applies when you pass mutable objects into functions. The function does not get a copy, it gets a reference to the same object. If the function modifies it, the change persists outside the function.

def add_interface(interface_list, interface_name):
    interface_list.append(interface_name)

interfaces = ["Gi0/1", "Gi0/2"]
add_interface(interfaces, "Gi0/3")

print(interfaces)

['Gi0/1', 'Gi0/2', 'Gi0/3']

Notice that we did not return anything from the function, and we did not reassign interfaces, but the original list outside the function changed. That is because the function was modifying the same list, not a copy of it.

This can be useful when you want it to happen, but it can also lead to bugs when you do not. The safest habit is to be aware of it, and to make a copy first if you do not want the original to change.

def add_interface(interface_list, interface_name):
    new_list = interface_list.copy()
    new_list.append(interface_name)
    return new_list

interfaces = ["Gi0/1", "Gi0/2"]
new_interfaces = add_interface(interfaces, "Gi0/3")

print(f"Original: {interfaces}")
print(f"New: {new_interfaces}")

Original: ['Gi0/1', 'Gi0/2']
New: ['Gi0/1', 'Gi0/2', 'Gi0/3']

Now the function builds a new list from the original, adds the new interface, and returns it. The caller can decide what to do with the result, and the original list is left untouched.