Python Error Handling

Throughout this series, you have seen Python errors quite a few times. We have run into TypeError when trying to add a string to a number, KeyError when accessing a missing dictionary key, ValueError when trying to convert bad text into an integer, and IndexError when trying to grab a list item that does not exist. Up until now, when you have hit an error, your script has simply crashed.

In small examples, that is fine. You see the error, you fix the code, and you move on. But in real scripts, errors are not always something you can prevent. You will be talking to devices that might not respond, parsing output that might not look the way you expect, reading files that might be missing, or making API calls that might time out. If your script crashes the moment any of this happens, it is not very useful, especially when you are running it across hundreds of devices.

This is where error handling comes in and helps. Python gives you a way to anticipate problems, catch them when they happen, and decide how to respond, all without your script crashing. In this chapter, we will look at how to use try and except blocks to handle errors gracefully, how to deal with different types of errors, and when you should and should not use error handling.

What is an Exception

When Python runs into a problem it cannot handle, it raises an exception. An exception is just Python's way of saying "something went wrong, and I do not know what to do." Unless you tell Python how to handle it, the exception stops your script and prints a traceback to the terminal.

Let's look at a familiar example.

print(int("hello"))

Output

Traceback (most recent call last):
  File "/Users/suresh/Documents/print_test.py", line 1, in <module>
    print(int("hello"))
          ~~~^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'hello'

That last line is the important part. It tells you two things, the type of exception (ValueError) and a message describing what went wrong. Different problems raise different types of exceptions. Here are a few you have already seen in this series.

• TypeError - you tried to do something with the wrong type of value, like adding a string to a number
• ValueError - the value itself is wrong for the operation, like trying to convert "hello" to an integer
• KeyError - you tried to access a key that does not exist in a dictionary
• IndexError - you tried to access an index that is out of range in a list

There are many more, but you do not need to memorise them. The names are usually self-explanatory, and when you hit one, the traceback tells you exactly which type it is.

The Problem with Crashing Scripts

Imagine you have written a script that connects to 100 devices and pulls the running config from each one. Now imagine that the 17th device is unreachable. Without error handling, your script crashes at device 17 and never gets to the other 83 devices. You have to fix the issue, restart the script, and hope nothing else fails.

This is the problem error handling solves. Instead of letting your script die at the first sign of trouble, you can wrap the risky parts in a try block, catch any exceptions that come up, and decide what to do next. Maybe you log the failure and move on. Maybe you retry the connection a few times. Maybe you skip that device entirely.

try and except

The basic syntax for handling exceptions is try and except. You put the code that might fail inside the try block, and the code that handles the failure inside the except block.

try:
    number = int("hello")
    print(number)
except:
    print("Something went wrong")

Output

Something went wrong

Python tried to run the code inside the try block. As soon as it hit int("hello"), a ValueError was raised. Instead of crashing, Python jumped to the except block and ran the code inside it.

Compare this to the same code without try/except, which would crash on that very first line. You can also have code after the try/except block, and it will keep running normally.

try:
    number = int("hello")
    print(number)
except:
    print("Could not convert to integer")

print("Script continues...")

Output

Could not convert to integer
Script continues...

Catching Specific Exceptions

Using a bare except like we did above catches every possible exception. Though that sounds convenient, but it is actually a bad habit. If you catch everything, you might end up hiding bugs that have nothing to do with what you were trying to handle.

A much better approach is to catch only the specific exception you are expecting.

try:
    number = int("hello")
    print(number)
except ValueError:
    print("That is not a valid number")

Output

That is not a valid number

Now we are only catching ValueError. If any other type of exception happens inside the try block, Python will not catch it, and the script will crash as usual, which is exactly what you want for bugs you did not plan for.

You can also grab the actual exception object using the as keyword. This is useful when you want to print or log the original error message.

try:
    number = int("hello")
except ValueError as error:
    print(f"Could not convert: {error}")

Output

Could not convert: invalid literal for int() with base 10: 'hello'

The variable error now holds the exception object, and printing it gives you the same message you would normally see in the traceback. This is much more informative when you are trying to figure out what went wrong.

Catching Multiple Exceptions

Sometimes the code inside your try block can raise more than one type of exception, and you want to handle each one differently. You can do this by adding multiple except blocks.

data = {"hostname": "r1_lnd_dc"}

try:
    ip = data["ip"]
    port = int(ip)
except KeyError:
    print("The IP key is missing from the dictionary")
except ValueError:
    print("The IP value is not a valid number")

Output

The IP key is missing from the dictionary

Python checks each except block in order and runs the first one that matches the exception. In this case, the KeyError was raised first (because "ip" does not exist in the dictionary), so only that block ran.

If you want to handle multiple exceptions the same way, you can group them into a tuple.

try:
    value = int("hello")
except (ValueError, TypeError) as error:
    print(f"Conversion failed: {error}")

Output

Conversion failed: invalid literal for int() with base 10: 'hello'

This is handy when several different errors should all be handled the same way.

else and finally

There are two more clauses you can add to a try/except block, else and finally. They are not used as often, but they are worth knowing about.

The else block runs only if the try block completes successfully, with no exception.

try:
    number = int("42")
except ValueError:
    print("Conversion failed")
else:
    print(f"Conversion succeeded, number is {number}")

Output

Conversion succeeded, number is 42

The finally block runs no matter what, whether the try succeeded, an exception was caught, or even an uncaught exception was raised. It is mainly used for cleanup, like closing files or disconnecting from devices.

try:
    number = int("hello")
except ValueError:
    print("Conversion failed")
finally:
    print("Cleaning up...")

Output

Conversion failed
Cleaning up...