Once you start working with Python, sooner or later you will come across having to work with CSV files. In this blog post, we will learn how to read, write and work with CSV files in Python.
A CSV is a comma-separated value file, which allows data to be saved in a tabular format. CSV files are plain-text files, making them easier to create, edit and move across various systems. Let's look at one of the CSV files - I first opened them as a text file and then open them as how you normally would open them in Excel.
ID, Hostname, Location 001, router_01, HQ 002, switch_01, DC 003, fw_02, HQ
As you can see above, each line in a CSV file represents a row. The elements are separated by 'commas' which is called the delimiter. The delimiter can be any single character, but
, is predominantly used.
Reading and Writing Files in Python
Before we dive into working with CSV files, let's have a look at the fundamentals of working with files in Python.
You can use Python's built-in
open() function to open a file. Once you finished working with that specific file, you should always close the file using the
close() function. When you open a file, you will also need to specify a mode.
specifying a mode (The default mode is
r- Read-Only mode
w- Write mode
a- Append mode
Let's say, I want to open a text file called 'my_file.txt' and read the contents. The Python code would look like the one below.
r = open('my_file.txt', 'r') #open a file which returns a 'File Object' print(r.read()) #Read the contents of the 'File Object' r.close() #Close the file
suresh@mac:~/Documents/python-projects/csv ⇒ python read_file.py Hello World!!!
Open()function returns a File Object that is simply another type of value in Python, just like lists and dictionaries.
- We stored the File Object in a variable called
- We then called the
read()method on the File Object to read the contents of the file.
- In the end, we closed the file by calling the
A better approach to opening a file is to use
with open() function. Using this approach, the file is automatically closed eliminating the need to manually close the file using
close() If we were to re-write the previous example using
with open(), it would look like the one below.
with open('my_file.txt', 'r') as r: print(r.read())
Back to CSV now - You can use the built-in
csv module to read and write CSV files in Python. First thing first, let's import the
csv module into your script.
Reading CSV Files
To read CSV files in Python, you can use the
csv.reader()method. Let's look at an example using the CSV file we created earlier.
import csv with open('test_csv.csv', 'r') as csv_file: reader_obj = csv.reader(csv_file) for row in reader_obj: print(row)
suresh@mac:~/Documents/python-projects/csv ⇒ python csv-reader.py ['ID', ' Hostname', ' Location'] ['001', ' router_01', ' HQ'] ['002', ' switch_01', ' DC'] ['003', ' fw_02', ' HQ']
- Please ensure the CSV file exists in the same directory as the Python script, else you will need to specify the full path.
with Open()function opens the file as a
csv.reader()method reads the file and returns an iterable reader object. The object is saved into a variable called
for loopiterate over the reader object and print out each row from the CSV file.
- Each line that we are printing out is a list with the values from the CSV file.
Writing CSV Files
To write to a CSV file in Python, we can use the (as you may have already guessed)
There are two methods to write to CSV files that are
writerow() takes one row, and
writerows() takes multiple rows at a time.
As you can see below,
writerow() takes an iterable of cells (single list) to write.
import csv with open('ghost_write.csv', 'w') as w_file: writer = csv.writer(w_file) writer.writerow(["ID", "Name", "Location"]) writer.writerow([1, "Max", "Netherland"]) writer.writerow([2, "Hamilton", "UK"]) writer.writerow([3, "Carlos", "Spain"])
writerows() on the other hand, takes a list of lists to write. This method is preferable if you have a large number of rows to write because it can write them out all at once.
import csv rows = [["ID", "Name", "Location"], [1, "Max", "Netherland"], [2, "Hamilton", "UK"], [3, "Carlos", "Spain"]] with open('ghost_write_rows.csv', 'w') as w_file: writer = csv.writer(w_file) writer.writerows(rows)
Append to a file
You can also open a file in append /
a mode where the write position will always be at the end of the file (an append).
import csv with open('ghost_write_rows.csv', 'a') as a_file: writer_append = csv.writer(a_file) writer_append.writerow([4, "Mick", "Germany"])
Reading CSV to a list in Python
Example - 1
We can read the CSV file into different Python data types such as list, dictionary or tuple. In this example, we will learn how to read a CSV file into a list. Let's use the CSV file from the previous example.
import csv with open('ghost_write_rows.csv', 'r') as reader_obj: csv_reader = csv.reader(reader_obj) new_list = list(csv_reader) print(new_list)
[['ID', 'Name', 'Location'], ['1', 'Max', 'Netherland'], ['2', 'Hamilton', 'UK'], ['3', 'Carlos', 'Spain'], ['4', 'Mick', 'Germany']]
As you can see above, the output contains a list of lists.
Let's say the CSV file only contains a single column and you want to convert each item in the row into a list. If we use the previous method, it will create a list of lists as shown below.
Single column CSV file
[['Slack'], ['Teams'], ['Zoom'], ['WebEx']]
What we wanted was, just a list of items, but instead, we got a list of lists. How can we just get a list of items? Well, you can use
list.append() method to get a list of items.
for loop iterates over each row and append each item to the list as shown below.
import csv rule_list =  with open('single_column.csv', 'r') as file_obj: reader_obj = csv.reader(file_obj) for row in reader_obj: rule_list.append(row) print(rule_list)
['Slack', 'Teams', 'Zoom', 'WebEx']
Python CSV module has more advanced features but I tried to cover the basics to get started. Please let me know in the comments if you have any questions or concerns.