MGU Computer Science Series — Programming with Python
Part V: Files, Data, and Practical Automation
Tables · CSV · Filtering · Cleaning · Type Conversion
Chapter 22 of 30
22
MGU Computer Science Series  ·  Programming with Python  ·  Library Textbook Edition

Chapter 22:
CSV Files and
Table Data

Reading and writing tables — rows, columns, headers, filtering, summarizing, cleaning, and converting real-world data with Python's csv module

■ Part V of VII ■

■ 10 Sections  •  Chapter 22 of 30 ■

Builds on Chapter 21 (files and text data) and Chapter 18 (dictionaries)

Chapter Introduction and Learning Objectives

Chapter 21 taught you how to read and write plain text files. That was a significant step. Your programs can now store information across runs, save reports, and load data from disk. But a lot of real-world data is not just lines of text. It is organized into tables: rows and columns, like a spreadsheet or a database. A student roster has rows for each student and columns for name, grade, and score. An inventory has rows for each product and columns for name, price, and quantity. A weather log has rows for each day and columns for date, high temperature, low temperature, and rainfall.

This chapter teaches you how to work with that kind of data in Python using CSV files. CSV stands for Comma-Separated Values. It is one of the most common and practical formats for exchanging table data between programs, spreadsheets, databases, and scripts.

Opening Question

A teacher has a spreadsheet of quiz scores for thirty students across three quizzes. She wants a program that computes each student's total and average, assigns a letter grade, prints a class report, and saves the results to a new file she can open later. None of this requires a database or a web application — it only requires a program that can read a table, do some arithmetic, and write a table back out. By the end of this chapter, you will be able to write exactly that kind of program.

10Sections
17Key Terms
5Debugging Bugs
30Total Chapters

Learning Objectives

By the end of this chapter, you should be able to:

  1. 1Explain what a table is and describe how rows, columns, headers, records, and fields relate to one another.
  2. 2Describe what a CSV file is and why it is a widely used format for exchanging table data.
  3. 3Read a CSV file using csv.reader() and csv.DictReader(), and explain when each is preferable.
  4. 4Write a CSV file using csv.writer() and csv.DictWriter(), including the header row.
  5. 5Filter rows of table data based on a condition, using both numeric and text comparisons.
  6. 6Summarize a column by computing its total, average, maximum, and minimum.
  7. 7Add a new computed column to every row in a table.
  8. 8Clean messy table data: stripping whitespace, normalizing case, and handling missing or invalid values.
  9. 9Explain why every value read from a CSV file is a string, and convert values correctly using int() and float().
  10. 10Combine loading, cleaning, filtering, summarizing, and saving into a complete, organized CSV-processing program.

Sections at a Glance

Sections 22.1 – 22.3
The Territory: Tables, CSV Files, and Vocabulary
Rows · Columns · Headers · Records · Fields

Tables as Data

Before writing any code, it helps to think clearly about what table data actually is. A table organizes information into rows and columns. Each row represents one record — one item, one person, one event, one measurement. Each column represents one attribute — one property that every record shares.

Here is a small example. Imagine you are tracking student quiz scores:

NameQuiz 1Quiz 2Quiz 3
Alice889279
Bob758190
Carmen958893
David627068

This table has four rows (one per student) and four columns (name and three quiz scores). Every row follows the same structure. Every column holds the same kind of value.

Tables are everywhere. Spreadsheets are tables. Database query results are tables. Reports from online systems often arrive as tables. When you export data from a bank, a school system, a weather station, or an inventory tool, you are often working with tables.

Python does not have a built-in "table" type the way a spreadsheet program does. But you already know structures that can represent table data:

Option 1
List of Lists
Each inner list is a row. Each position in the inner list is a column. You refer to columns by index: row[0], row[1].
Option 2
List of Dictionaries
Each dictionary is a row. Each key is a column name. You refer to columns by name: row["Name"]. Easier to read for most beginner work.
The Key Insight

Reading a CSV file means turning table data stored on disk into one of these Python structures. Writing a CSV file means turning one of these Python structures back into table data on disk. Both approaches work; this chapter uses both, but leans on the list-of-dictionaries approach because referring to columns by name is clearer than referring to them by position.

CSV Files

CSV stands for Comma-Separated Values. A CSV file is a plain text file where each line represents one row of a table, and the values in each row are separated by commas.

Here is what the student quiz table from the previous section looks like as a CSV file:

students.csv
Name,Quiz 1,Quiz 2,Quiz 3
Alice,88,92,79
Bob,75,81,90
Carmen,95,88,93
David,62,70,68

There is no formatting here. No bold text, no borders, no colors. Just values separated by commas, one row per line. The first line is typically the header row — the column names. Every line after that is a data row.

CSV files are popular because:

  • They are plain text, so any program can read or write them.
  • They are simple. No special software is needed to create or inspect them.
  • Spreadsheets like Excel and Google Sheets can open and export CSV files.
  • Databases can import and export CSV files.
  • Python has built-in tools for reading and writing CSV.

CSV files do have limitations. They only store flat table data — no formulas, no multiple sheets, no data types beyond strings. But for a large range of practical tasks, they are exactly the right tool.

A Note on CSV Variations

Most CSV files use commas as separators, but some use semicolons or tabs. Some files wrap text values in quotation marks. Some files include a header row; some do not. Python's csv module handles most of these variations gracefully, and you will learn how to work with the most common ones.

To open a CSV file and look at its contents, you can open it in any plain text editor. You can also open it in a spreadsheet program if you want to see it in table form. But in Python, you will always treat it as a text file and process it line by line.

Rows, Columns, and Headers

Let us be precise about vocabulary before moving to code.

TermMeaning
RowOne line of data in the CSV file (after the header). Each row represents one record.
ColumnOne field position across all rows. In the student example, Quiz 1 is a column.
Header rowThe first line of the CSV file. Contains the column names, not data.
RecordOne complete row of data — for example, a single student's name and three scores.
FieldOne value within a record, such as "Alice" or 88.

When Python's csv module reads a CSV file, it can give you access to each row as either:

  • A list: ['Alice', '88', '92', '79']
  • A dictionary: {'Name': 'Alice', 'Quiz 1': '88', 'Quiz 2': '92', 'Quiz 3': '79'}

The list approach gives you columns by position (index 0, 1, 2, ...). The dictionary approach gives you columns by name. For most programs, dictionaries are easier to work with because you do not need to remember which column number holds which field.

Important

Every value read from a CSV file arrives as a string, even if it looks like a number. '88' is a string, not an integer. If you want to do arithmetic, you must convert it first. You will practice this carefully in Section 22.9.

Sections 22.4 – 22.5
The Tools: Reading and Writing CSV Files
csv.reader · csv.DictReader · csv.writer · csv.DictWriter

Reading CSV Files

Python's standard library includes the csv module. You import it at the top of your program:

Python
import csv

To read a CSV file, you open it with the standard open() function and then pass the file object to csv.reader() or csv.DictReader().

Reading with csv.reader()

csv.reader() gives you each row as a list of strings. Here is a complete example. Assume the file students.csv exists and contains the rows shown in Section 22.2.

Python
import csv

with open("students.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)
Output
['Name', 'Quiz 1', 'Quiz 2', 'Quiz 3']
['Alice', '88', '92', '79']
['Bob', '75', '81', '90']
['Carmen', '95', '88', '93']
['David', '62', '70', '68']

Notice that the header row is included. Each row is a list. All values are strings. If you want to skip the header row, you can call next() on the reader to advance past it:

Python
import csv

with open("students.csv", "r") as file:
    reader = csv.reader(file)
    next(reader)    # skip the header row
    for row in reader:
        print(row)
Output
['Alice', '88', '92', '79']
['Bob', '75', '81', '90']
['Carmen', '95', '88', '93']
['David', '62', '70', '68']

To access a specific column, use the index. row[0] is the name, row[1] is Quiz 1, row[2] is Quiz 2, row[3] is Quiz 3.

Python
import csv

with open("students.csv", "r") as file:
    reader = csv.reader(file)
    next(reader)
    for row in reader:
        name = row[0]
        score = row[1]
        print(f"{name} scored {score} on Quiz 1.")
Output
Alice scored 88 on Quiz 1.
Bob scored 75 on Quiz 1.
Carmen scored 95 on Quiz 1.
David scored 62 on Quiz 1.

Reading with csv.DictReader()

csv.DictReader() is usually easier to use. It automatically reads the first row as the header and gives you each subsequent row as a dictionary where the keys are the column names.

Python
import csv

with open("students.csv", "r") as file:
    reader = csv.DictReader(file)
    for row in reader:
        print(row)
Output
{'Name': 'Alice', 'Quiz 1': '88', 'Quiz 2': '92', 'Quiz 3': '79'}
{'Name': 'Bob', 'Quiz 1': '75', 'Quiz 2': '81', 'Quiz 3': '90'}
{'Name': 'Carmen', 'Quiz 1': '95', 'Quiz 2': '88', 'Quiz 3': '93'}
{'Name': 'David', 'Quiz 1': '62', 'Quiz 2': '70', 'Quiz 3': '68'}

Now you access values by name:

Python
import csv

with open("students.csv", "r") as file:
    reader = csv.DictReader(file)
    for row in reader:
        name = row["Name"]
        q1 = row["Quiz 1"]
        print(f"{name}: Quiz 1 = {q1}")
Output
Alice: Quiz 1 = 88
Bob: Quiz 1 = 75
Carmen: Quiz 1 = 95
David: Quiz 1 = 62
Why DictReader Is Usually Better

Accessing values by name (row["Name"]) is much clearer than accessing them by index (row[0]). For most of the work in this chapter, you will use DictReader.

Loading Rows into a List

Often you want to read all the rows into a list so you can work with them after the file is closed.

Python
import csv

def load_students(filename):
    students = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            students.append(row)
    return students

students = load_students("students.csv")
print(f"Loaded {len(students)} students.")
print(students[0])
Output
Loaded 4 students.
{'Name': 'Alice', 'Quiz 1': '88', 'Quiz 2': '92', 'Quiz 3': '79'}

This pattern — open the file, read all rows into a list, close the file, then process the list — is one of the most common patterns in CSV work.

Writing CSV Files

You write a CSV file using csv.writer() or csv.DictWriter().

Writing with csv.writer()

csv.writer() takes a file object and lets you write rows as lists.

Python
import csv

rows = [
    ["Name", "Score", "Grade"],
    ["Alice", 92, "A"],
    ["Bob", 81, "B"],
    ["Carmen", 95, "A"],
    ["David", 68, "D"],
]

with open("results.csv", "w", newline="") as file:
    writer = csv.writer(file)
    for row in rows:
        writer.writerow(row)
Don't Forget newline=""

The newline="" argument is important when writing CSV files on Windows. Without it, you may get extra blank lines in the output file. On any platform, it is good practice to include it.

After running this, results.csv will contain:

results.csv
Name,Score,Grade
Alice,92,A
Bob,81,B
Carmen,95,A
David,68,D

You can also write all rows at once using writerows():

Python
writer.writerows(rows)

Writing with csv.DictWriter()

If your data is stored as a list of dictionaries, csv.DictWriter() is the better choice. You specify the column names (the fieldnames) when you create the writer, and then write each row as a dictionary.

Python
import csv

students = [
    {"Name": "Alice", "Score": 92, "Grade": "A"},
    {"Name": "Bob", "Score": 81, "Grade": "B"},
    {"Name": "Carmen", "Score": 95, "Grade": "A"},
    {"Name": "David", "Score": 68, "Grade": "D"},
]

fieldnames = ["Name", "Score", "Grade"]

with open("results.csv", "w", newline="") as file:
    writer = csv.DictWriter(file, fieldnames=fieldnames)
    writer.writeheader()        # writes the header row
    for student in students:
        writer.writerow(student)

writer.writeheader() writes the column names as the first row. Then writer.writerow() writes each dictionary as a data row. The fieldnames list controls which columns appear and in what order. If a dictionary has a key that is not in fieldnames, it is ignored.

A Complete Read-Process-Write Pattern

Here is the full pattern you will use throughout this chapter: read a CSV file, process the data, write the results to a new CSV file.

Python
import csv

def load_records(filename):
    records = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            records.append(row)
    return records

def save_records(records, fieldnames, filename):
    with open(filename, "w", newline="") as file:
        writer = csv.DictWriter(file, fieldnames=fieldnames)
        writer.writeheader()
        for record in records:
            writer.writerow(record)

# Load
students = load_records("students.csv")

# Process (add a total column — see Section 22.7)
# ...

# Save
# save_records(students, ["Name", "Quiz 1", "Quiz 2", "Quiz 3", "Total"], "results.csv")
Design Habit

Separating loading, processing, and saving into functions keeps your program organized and easier to test.

Sections 22.6 – 22.7
Working with Table Data: Filtering and Summarizing
Conditions · Totals · Averages · Computed Columns

Filtering Rows

Filtering means selecting only the rows that meet a condition. You read all the rows, then keep only the ones you want. The basic pattern is:

Pattern
results = []
for row in all_rows:
    if <condition>:
        results.append(row)

Example: Students Who Passed

Assume you want only students whose Quiz 1 score is 75 or higher.

Python
import csv

def load_students(filename):
    students = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            students.append(row)
    return students

def filter_passing(students, threshold):
    passing = []
    for student in students:
        score = int(student["Quiz 1"])
        if score >= threshold:
            passing.append(student)
    return passing

students = load_students("students.csv")
passing = filter_passing(students, 75)

for s in passing:
    print(s["Name"], s["Quiz 1"])
Output
Alice 88
Bob 75
Carmen 95

David had a score of 62 and was not included. Notice the int() conversion on the line score = int(student["Quiz 1"]). All values from a CSV file arrive as strings. You must convert them before comparing them numerically. You will practice this more carefully in Section 22.9.

Example: Filtering by a Text Value

You can also filter on string columns. Suppose your CSV file has a City column and you want only rows where the city is "Portland":

Python
portland_rows = []
for row in all_rows:
    if row["City"] == "Portland":
        portland_rows.append(row)

Using a Function to Filter

Writing filtering logic as a separate function makes it reusable and easier to test:

Python
def filter_by_city(records, city):
    result = []
    for record in records:
        if record["City"] == city:
            result.append(record)
    return result
Reusability

You can call filter_by_city() with different city names without rewriting the loop. This is the same idea you practiced with helper functions in Chapter 14.

Summarizing Columns

Summarizing means computing a value across all rows in a column: a total, an average, a maximum, a minimum, or a count.

Total, Average, Maximum, Minimum

Python
def column_total(records, column_name):
    total = 0
    for record in records:
        total += int(record[column_name])
    return total

def column_average(records, column_name):
    if len(records) == 0:
        return 0
    total = 0
    for record in records:
        total += float(record[column_name])
    return total / len(records)

def column_max(records, column_name):
    values = []
    for record in records:
        values.append(float(record[column_name]))
    return max(values)

def column_min(records, column_name):
    values = []
    for record in records:
        values.append(float(record[column_name]))
    return min(values)

Putting It Together

Here is a complete example that loads the student file and reports a summary of Quiz 1 scores:

Python
import csv

def load_students(filename):
    students = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            students.append(row)
    return students

def summarize_column(students, column_name):
    values = [float(s[column_name]) for s in students]
    total = sum(values)
    average = total / len(values)
    highest = max(values)
    lowest = min(values)
    return total, average, highest, lowest

students = load_students("students.csv")
total, average, highest, lowest = summarize_column(students, "Quiz 1")

print(f"Quiz 1 Summary")
print(f"  Total:   {total:.1f}")
print(f"  Average: {average:.1f}")
print(f"  Highest: {highest:.1f}")
print(f"  Lowest:  {lowest:.1f}")
Output
Quiz 1 Summary
  Total:   320.0
  Average: 80.0
  Highest: 95.0
  Lowest:  62.0

This uses a list comprehension on the line values = [float(s[column_name]) for s in students]. If you prefer to avoid list comprehensions, the equivalent loop is:

Python — equivalent loop
values = []
for s in students:
    values.append(float(s[column_name]))
Style Note

Both forms work. The list comprehension is more compact, but the loop is easier to read when you are just learning. Use whichever form you can explain clearly.

Adding a Computed Column

Sometimes you want to add a new column to each row based on a calculation. For example, you might want to add a Total column that sums all three quiz scores.

Python
def add_total(students):
    for student in students:
        q1 = int(student["Quiz 1"])
        q2 = int(student["Quiz 2"])
        q3 = int(student["Quiz 3"])
        student["Total"] = q1 + q2 + q3
    return students

students = load_students("students.csv")
students = add_total(students)

for s in students:
    print(s["Name"], s["Total"])
Output
Alice 259
Bob 246
Carmen 276
David 200

Because each row is a dictionary and dictionaries are mutable, you can add a new key directly. The new "Total" key is added to each existing dictionary.

Sections 22.8 – 22.10
Making Data Trustworthy: Cleaning, Converting, and Real Projects
Whitespace · Missing Values · Type Conversion · Combining Files

Cleaning Table Data

Real CSV files are often messy. Values may have extra spaces, inconsistent capitalization, missing fields, or the wrong format. Cleaning data means fixing these problems before processing.

Stripping Whitespace

A common problem is that values have leading or trailing spaces: " Alice" instead of "Alice", or " 88 " instead of "88". The .strip() method removes them.

Python
name = "  Alice  "
clean_name = name.strip()
print(repr(clean_name))   # 'Alice'

You can strip every field when loading:

Python
def load_clean_students(filename):
    students = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            clean_row = {}
            for key, value in row.items():
                clean_row[key.strip()] = value.strip()
            students.append(clean_row)
    return students

Normalizing Case

Names or category values may use inconsistent capitalization: "alice", "ALICE", "Alice". You can normalize them:

Python
name = row["Name"].strip().title()

.title() capitalizes the first letter of each word. .lower() converts everything to lowercase. .upper() converts to uppercase.

Handling Missing Values

Sometimes a field is empty. In a CSV file this looks like two commas with nothing between them: Alice,,92,79. Python's csv module reads this as an empty string "".

You should decide how to handle missing values before processing. Common choices are to skip the row, use a default value (such as 0), or flag it:

Python
def safe_int(value, default=0):
    value = value.strip()
    if value == "":
        return default
    return int(value)

Then use safe_int(row["Quiz 1"]) instead of int(row["Quiz 1"]).

Checking for Valid Values

Sometimes values are present but wrong — a negative score, a date in the wrong format, or a city name that should not have numbers in it. You can add validation checks:

Python
def is_valid_score(value):
    try:
        score = int(value)
        return 0 <= score <= 100
    except ValueError:
        return False
Looking Ahead

You will learn more about try and except in Chapter 23. For now, know that this approach lets your program check whether a value can be converted before converting it.

A Worked Cleaning Example

Suppose your CSV file has a few problems: names have extra spaces, some scores are missing, and one score is not a number.

students.csv (messy)
Name, Quiz 1, Quiz 2, Quiz 3
 Alice , 88, 92, 79
Bob,75,,90
Carmen,95,88,not_recorded
David,62,70,68

Here is a cleaning function that handles these issues:

Python
import csv

def safe_int(value, default=0):
    value = value.strip()
    if value == "":
        return default
    try:
        return int(value)
    except ValueError:
        return default

def load_and_clean(filename):
    students = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            clean = {
                "Name": row["Name"].strip(),
                "Quiz 1": safe_int(row["Quiz 1"]),
                "Quiz 2": safe_int(row["Quiz 2"]),
                "Quiz 3": safe_int(row["Quiz 3"]),
            }
            students.append(clean)
    return students

students = load_and_clean("students.csv")
for s in students:
    print(s)
Output
{'Name': 'Alice', 'Quiz 1': 88, 'Quiz 2': 92, 'Quiz 3': 79}
{'Name': 'Bob', 'Quiz 1': 75, 'Quiz 2': 0, 'Quiz 3': 90}
{'Name': 'Carmen', 'Quiz 1': 95, 'Quiz 2': 88, 'Quiz 3': 0}
{'Name': 'David', 'Quiz 1': 62, 'Quiz 2': 70, 'Quiz 3': 68}

Bob's missing Quiz 2 became 0. Carmen's "not_recorded" Quiz 3 became 0. Spaces were stripped from Alice's name. This is not always the right behavior — in a real system you might want to flag bad data differently — but it is a reasonable starting point.

Cleaning Is Not Optional

If you skip cleaning and process messy values, your calculations will be wrong or your program will crash. Always inspect a sample of your data before writing processing code.

Converting Between Strings and Numbers

This section deserves its own space because the string-to-number conversion problem is the most common source of bugs in CSV programs.

Remember This

Every value read from a CSV file is a string. Even "88" is a string. You cannot do arithmetic on strings.

Wrong
score = row["Quiz 1"]     # score is "88", a string
total = score + 10        # TypeError or "8810", not 98
Correct
score = int(row["Quiz 1"])     # score is 88, an integer
total = score + 10              # total is 98

int() versus float()

Use int() when the value should be a whole number: counts, scores, ages, years, quantities. Use float() when the value may have a decimal: prices, measurements, temperatures, percentages.

Python
price = float(row["Price"])      # "12.99" → 12.99
quantity = int(row["Quantity"])  # "5" → 5
total = price * quantity         # 64.95

What Happens When Conversion Fails

If the value cannot be converted, Python raises a ValueError:

Python
int("hello")     # ValueError: invalid literal for int() with base 10: 'hello'
float("")        # ValueError: could not convert string to float: ''
int("9.5")       # ValueError: invalid literal for int() with base 10: '9.5'
Watch the Last Example

"9.5" is not a valid integer string. You must use float("9.5") and then convert to int if needed: int(float("9.5")) gives 9 (truncated, not rounded).

Converting Back to Strings for Writing

When you write numbers back to a CSV file, Python's csv module handles the conversion automatically. You do not need to call str() before writing. But if you are building strings manually (for printing or logging), you will need str() or an f-string.

Python
score = 88
line = f"Score: {score}"       # works fine

A Conversion Pattern Worth Memorizing

Load as strings. Convert when you need numbers. Store converted values if you will use them multiple times.

Python
def load_scores(filename):
    records = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            record = {
                "name": row["Name"].strip(),
                "q1": int(row["Quiz 1"].strip()),
                "q2": int(row["Quiz 2"].strip()),
                "q3": int(row["Quiz 3"].strip()),
            }
            records.append(record)
    return records

After loading, every q1, q2, and q3 value is already an integer. Your processing code never needs to convert again.

CSV Data in Real Projects

CSV files appear in almost every domain. Before finishing this chapter, it is worth seeing several realistic patterns.

Pattern 1: Report Generation

Read a CSV file, compute summaries, print a formatted report.

Python
import csv

def load_products(filename):
    products = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            products.append({
                "name": row["Product"].strip(),
                "price": float(row["Price"].strip()),
                "quantity": int(row["Quantity"].strip()),
            })
    return products

def print_inventory_report(products):
    print(f"{'Product':<20} {'Price':>8} {'Qty':>6} {'Value':>10}")
    print("-" * 46)
    total_value = 0
    for p in products:
        value = p["price"] * p["quantity"]
        total_value += value
        print(f"{p['name']:<20} {p['price']:>8.2f} {p['quantity']:>6} {value:>10.2f}")
    print("-" * 46)
    print(f"{'Total value':>36} {total_value:>10.2f}")

products = load_products("inventory.csv")
print_inventory_report(products)

Pattern 2: Filtering and Saving a Subset

Read a CSV file, keep only matching rows, save to a new file.

Python
import csv

def load_csv(filename):
    records = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        fieldnames = reader.fieldnames
        for row in reader:
            records.append(row)
    return records, fieldnames

def save_csv(records, fieldnames, filename):
    with open(filename, "w", newline="") as file:
        writer = csv.DictWriter(file, fieldnames=fieldnames)
        writer.writeheader()
        for record in records:
            writer.writerow(record)

records, fieldnames = load_csv("employees.csv")

# Keep only employees in the Engineering department
engineering = [r for r in records if r["Department"].strip() == "Engineering"]

save_csv(engineering, fieldnames, "engineering_team.csv")
print(f"Saved {len(engineering)} engineering records.")

Pattern 3: Combining Two CSV Files

Read two CSV files and combine their data. For example, one file has student names and IDs, another has student IDs and scores:

Python
import csv

def load_by_id(filename, id_column):
    records = {}
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            key = row[id_column].strip()
            records[key] = row
    return records

students = load_by_id("students.csv", "StudentID")
scores = load_by_id("scores.csv", "StudentID")

# Combine: attach scores to each student
for sid, student in students.items():
    if sid in scores:
        student["Score"] = scores[sid]["Score"]
    else:
        student["Score"] = "N/A"

# Print combined data
for sid, student in students.items():
    print(f"{student['Name']}: {student['Score']}")
A Foundational Technique

Using a dictionary keyed by a unique ID to look up matching records is one of the foundational operations in data work. You will see it again in database courses.

Pattern 4: Counting and Grouping

Count how many records belong to each category.

Python
import csv

def count_by_column(records, column_name):
    counts = {}
    for record in records:
        value = record[column_name].strip()
        if value in counts:
            counts[value] += 1
        else:
            counts[value] = 1
    return counts

records, _ = load_csv("survey.csv")
department_counts = count_by_column(records, "Department")

for department, count in sorted(department_counts.items()):
    print(f"{department}: {count}")

This is the counting with dictionaries pattern from Chapter 18, now applied to CSV data.

Thinking About Scale

The programs in this chapter work well for small CSV files — hundreds or a few thousand rows. For much larger files (millions of rows), you would use tools like Python's pandas library, a database, or streaming processing. Those tools are beyond this book's scope, but they are natural next steps after you are comfortable with what this chapter teaches.

For now, understand that the patterns here — load, clean, filter, summarize, transform, save — are the same patterns those larger tools use. Learning them at small scale with Python's csv module builds the reasoning skills you need for bigger tools later.

Student Grade Analyzer

Let us put everything together in one complete program. This program reads a CSV file of student quiz scores, adds a total and average column, assigns a letter grade, prints a report, and saves the results to a new file.

The Input File

quiz_scores.csv
Name,Quiz 1,Quiz 2,Quiz 3
Alice,88,92,79
Bob,75,81,90
Carmen,95,88,93
David,62,70,68
Elise,84,77,85

The Program

Python — grade_analyzer.py
import csv

# ── Loading ──────────────────────────────────────────────────

def load_scores(filename):
    """Load quiz scores from a CSV file. Returns a list of dicts."""
    students = []
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            students.append({
                "name":  row["Name"].strip(),
                "quiz1": int(row["Quiz 1"].strip()),
                "quiz2": int(row["Quiz 2"].strip()),
                "quiz3": int(row["Quiz 3"].strip()),
            })
    return students

# ── Processing ───────────────────────────────────────────────

def assign_grade(average):
    """Return a letter grade for a numeric average."""
    if average >= 90:
        return "A"
    elif average >= 80:
        return "B"
    elif average >= 70:
        return "C"
    elif average >= 60:
        return "D"
    else:
        return "F"

def add_summary_columns(students):
    """Add total, average, and grade to each student record."""
    for student in students:
        total = student["quiz1"] + student["quiz2"] + student["quiz3"]
        average = total / 3
        student["total"]   = total
        student["average"] = round(average, 1)
        student["grade"]   = assign_grade(average)
    return students

# ── Reporting ────────────────────────────────────────────────

def print_report(students):
    """Print a formatted grade report."""
    header = f"{'Name':<10} {'Q1':>4} {'Q2':>4} {'Q3':>4} {'Total':>6} {'Avg':>6} {'Grade':>6}"
    print(header)
    print("-" * len(header))
    for s in students:
        print(
            f"{s['name']:<10} {s['quiz1']:>4} {s['quiz2']:>4} {s['quiz3']:>4}"
            f" {s['total']:>6} {s['average']:>6} {s['grade']:>6}"
        )

def class_summary(students):
    """Print class-wide statistics."""
    averages = [s["average"] for s in students]
    class_avg = sum(averages) / len(averages)
    highest   = max(averages)
    lowest    = min(averages)
    print(f"\nClass average: {class_avg:.1f}")
    print(f"Highest average: {highest:.1f}")
    print(f"Lowest average:  {lowest:.1f}")

# ── Saving ───────────────────────────────────────────────────

def save_results(students, filename):
    """Save processed student data to a CSV file."""
    fieldnames = ["name", "quiz1", "quiz2", "quiz3", "total", "average", "grade"]
    with open(filename, "w", newline="") as file:
        writer = csv.DictWriter(file, fieldnames=fieldnames)
        writer.writeheader()
        for student in students:
            writer.writerow(student)

# ── Main ─────────────────────────────────────────────────────

def main():
    students = load_scores("quiz_scores.csv")
    students = add_summary_columns(students)
    print_report(students)
    class_summary(students)
    save_results(students, "grade_results.csv")
    print("\nResults saved to grade_results.csv")

main()

Output

Console Output
Name        Q1   Q2   Q3  Total    Avg  Grade
-----------------------------------------------
Alice       88   92   79    259   86.3      B
Bob         75   81   90    246   82.0      B
Carmen      95   88   93    276   92.0      A
David       62   70   68    200   66.7      D
Elise       84   77   85    246   82.0      B

Class average: 81.8
Highest average: 92.0
Lowest average:  66.7

Results saved to grade_results.csv

Tracing the Data Flow

It is worth tracing what happens to one piece of data as it moves through this program. The string "88" in the CSV file becomes the integer 88 during load_scores() when int(row["Quiz 1"].strip()) runs. It is stored as student["quiz1"]. During add_summary_columns(), it is added to quiz2 and quiz3 to compute total. The average is computed from total / 3. The string "B" is computed by assign_grade(86.3). During save_results(), the integer 88 is written back as the text 88 in the output CSV.

The Heart of CSV Programming

Text in, structured data processed, text out. This flow — read strings, convert and compute, write results — is the pattern behind nearly every CSV program you will write.

Chapter Summary

Key Terms

CSV
Comma-Separated Values. A plain text file format where each line represents a table row and values are separated by commas.
table
Data organized into rows and columns, where each row is a record and each column is an attribute.
row
One line of data in a CSV file, representing one record.
column
One field position across all rows in a table.
header row
The first line of a CSV file, containing the column names.
record
One complete row of data.
field
One value within a record.
csv module
Python's standard library module for reading and writing CSV files.
csv.reader()
Reads CSV rows as lists of strings.
csv.DictReader()
Reads CSV rows as dictionaries, using the header row as keys.
csv.writer()
Writes rows as lists to a CSV file.
csv.DictWriter()
Writes rows as dictionaries to a CSV file.
writeheader()
A DictWriter method that writes the header row.
filtering
Selecting only the rows that meet a condition.
summarizing
Computing aggregate values (total, average, maximum, minimum) across a column.
data cleaning
Fixing problems in raw data: whitespace, inconsistent case, missing values, invalid values.
type conversion
Converting a value from one type to another, such as int("88") or float("12.99").
PracticeCSV Reading Practice

Questions 1–5. Study the following CSV file contents, then answer each question.

inventory_sample.csv
ProductID,Name,Category,Price,Quantity
101,Spiral Notebook,Supplies,3.49,200
102,Blue Pen,Supplies,1.25,450
103,Desk Lamp,Electronics,24.99,75
104,USB Cable,Electronics,8.99,130
105,Sticky Notes,Supplies,2.75,300
  1. How many data rows does this CSV file have?
  2. What is the value in column Category of the third data row?
  3. If you read this file with csv.DictReader(), what keys would each row dictionary have?
  4. What Python expression would give you the price of the first product, as a float, from a row named row?
  5. A program reads this file and prints row["Price"] for the first data row. What does it print, and what is its type?
PracticeTable Processing Questions

Questions 6–10.

  1. You have a list of dictionaries called products, each with keys "Name", "Price", and "Quantity". Write a loop that prints the name of every product whose price is less than 5.00. You must convert the price before comparing.
  2. Write a function called total_quantity(products) that takes the list of product dictionaries and returns the sum of all "Quantity" values as an integer.
  3. Write a function called most_expensive(products) that returns the dictionary of the product with the highest price.
  4. You have a CSV file where one column is called "InStock" and contains the values "Yes" or "No". Write a filtering function that returns only the rows where "InStock" is "Yes".
  5. A program reads a CSV file and computes an average. The result is much larger than expected. You suspect type conversion is the problem. Explain what is likely happening and how to fix it.
PracticeData Cleaning Scenarios

Scenarios 11–15. For each scenario, describe what the problem is and write or describe code to fix it.

  1. You read a CSV file and find that the "Name" column has values like " alice ", "BOB", and "Carmen ". You want all names to be properly capitalized and free of extra spaces.
  2. Some rows in your CSV file have an empty string "" in the "Score" column. Your program crashes with a ValueError when it tries to convert the empty string to an integer. How do you handle this?
  3. You are reading a CSV file that was exported from a different program. The numbers in the "Price" column are formatted as "$12.99" and "$3.49". How do you convert these to floats?
  4. Your CSV file has a "Date" column with values in two formats: some rows use "2024-03-15" and others use "03/15/2024". For now, you only want to keep rows that use the "YYYY-MM-DD" format. How do you filter for those rows?
  5. You read a file and discover that the first column header has a hidden space: it reads as " Name" instead of "Name". When you try row["Name"], you get a KeyError. How do you fix this?
PracticeDebugging Practice

Each code block below contains an error. Identify the error and correct it.

Bug 1
import csv

with open("data.csv", "r") as file:
    reader = csv.DictReader(file)
    for row in reader:
        score = row["Score"] + 10
        print(score)
Bug 2
import csv

rows = [
    {"Name": "Alice", "Score": 92},
    {"Name": "Bob", "Score": 85},
]

with open("output.csv", "w") as file:
    writer = csv.DictWriter(file, fieldnames=["Name", "Score"])
    for row in rows:
        writer.writerow(row)
The file is created but the header row is missing. What should be added?
Bug 3
import csv

def get_average(filename, column):
    total = 0
    count = 0
    with open(filename, "r") as file:
        reader = csv.DictReader(file)
        for row in reader:
            total += row[column]
            count += 1
    return total / count
Bug 4
import csv

with open("students.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerow("Name", "Grade")
    writer.writerow("Alice", "A")
Hint: writerow takes one argument. What should it be?
Bug 5
import csv

records = []
with open("inventory.csv") as file:
    for row in file:
        records.append(row)

print(records[0]["ProductName"])
This program reads a CSV file without using the csv module. What is wrong with records[0]["ProductName"]?
PracticeShort Programming Tasks
  1. Task 1. Write a program that reads a CSV file called temperatures.csv with columns Date and High. It should print the date and temperature of the hottest day.
  2. Task 2. Write a function called count_category(records, column, value) that takes a list of row dictionaries, a column name, and a target value, and returns the count of rows where that column equals that value.
  3. Task 3. Write a function called add_column(records, column_name, default_value) that adds a new key to every dictionary in the list, set to the default value. This is useful for adding a computed column before filling it in.
  4. Task 4. Write a program that reads a CSV file with columns Name, Score1, Score2, and Score3. For each row, it should print the student's name and their highest individual score.
  5. Task 5. Write a function called csv_to_list_of_dicts(filename) that reads any CSV file and returns a list of dictionaries, with all string values stripped of whitespace. This should work for any CSV file regardless of its column names.
  6. Task 6. Write a program that reads a CSV file with a Department column and produces a summary: how many employees are in each department, and what is the average salary in each department.
  7. Task 7. Write a program that reads a CSV file, removes any row where any field is an empty string, and saves the cleaned data to a new file with _clean added to the filename.
Mini-ProjectCSV Gradebook Analyzer

Build a complete gradebook analyzer for a class CSV file.

Requirements

Your program should read a CSV file called gradebook.csv. The file has the following structure:

gradebook.csv
StudentID,Name,HW1,HW2,HW3,Midterm,Final
1001,Alice,90,85,92,88,91
1002,Bob,78,82,75,80,77
1003,Carmen,95,98,93,97,94
1004,David,60,65,70,68,72
1005,Elise,85,80,88,84,86

Your program must:

  1. Load the gradebook from the CSV file, converting all numeric columns to integers.
  2. Compute a weighted total for each student using these weights: each homework assignment counts 10% (3 assignments = 30% total), midterm counts 30%, and final counts 40%.
  3. Assign a letter grade based on the weighted total: 90–100 is A, 80–89 is B, 70–79 is C, 60–69 is D, and below 60 is F.
  4. Print a formatted report showing each student's name, individual scores, weighted total, and letter grade.
  5. Print a class summary: class average weighted total, highest total, lowest total, and grade distribution (how many A, B, C, D, F grades).
  6. Save the results to a file called gradebook_results.csv with columns: StudentID, Name, HW1, HW2, HW3, Midterm, Final, WeightedTotal, Grade.

Extensions

If you finish the main requirements, try these additions:

  • Add a function that prints the names of students who improved from midterm to final (Final greater than Midterm).
  • Add a function that prints the assignment with the lowest class average.
  • Allow the user to enter a student name and look up their record interactively.
  • Handle a version of the file where some homework scores are missing (empty strings), treating missing scores as 0.

Planning Tips

Before writing code, sketch the data structure. Each student record will be a dictionary. What keys will it have after loading? What keys will it have after processing? Write and test one function at a time. Test load_gradebook() before writing compute_weighted_total(). Test compute_weighted_total() with one record before applying it to all records. Use the round() function or format with .1f to avoid printing many decimal places.

CumulativeMixed Review

These questions draw on skills from earlier chapters.

  1. Review 1. Write a function that takes a list of numbers and returns the second-largest value. (Chapter 17)
  2. Review 2. You have a dictionary called scores where keys are student names and values are lists of quiz scores. Write code that prints each student's name and their average score. (Chapters 18–19)
  3. Review 3. A program reads a CSV file and appends a new row to a list. After the loop, the programmer checks records[-1] to see the last record. What does -1 as a list index mean? (Chapter 17)
  4. Review 4. Write a function called clean_text(text) that strips whitespace, converts to title case, and returns the result. Then show how you would apply it to the "Name" field of every row in a list of dictionaries. (Chapter 16)
  5. Review 5. You want to open a file but the file might not exist. You have not yet learned try/except formally, but you know it exists. What chapter will cover this? What would happen if your program tries to open a missing file right now? (Preview of Chapter 23)
ReviewReview Questions
  1. What does CSV stand for, and what does a CSV file contain?
  2. What is the difference between csv.reader() and csv.DictReader()? When would you prefer each?
  3. Why must you include newline="" when opening a file for writing with csv.writer()?
  4. All values read from a CSV file have the same Python type. What type is that?
  5. What is the purpose of writer.writeheader() in a csv.DictWriter program?
  6. A CSV file has 100 data rows and a header. How many times does the body of a for row in reader: loop execute when using DictReader?
  7. You are filtering a list of row dictionaries and want to keep rows where the "Score" column is greater than 80. Write the condition you would use inside the if statement, including any necessary type conversion.
  8. Describe two common data quality problems you might find in a CSV file and how you would fix each one.
  9. What does the .strip() method do, and why is it especially useful when reading CSV data?
  10. You want to add a "Total" key to every dictionary in a list called records. Write a loop that does this, where "Total" is the sum of "Score1" and "Score2" (already stored as integers).
Chapter 22 Checklist
Use this to confirm your understanding before moving to Chapter 23
Concepts
  • I can explain what a table is and describe rows, columns, headers, records, and fields.
  • I can describe what a CSV file is and why it is widely used.
  • I understand that every value read from a CSV file arrives as a string.
  • I can explain the difference between csv.reader() and csv.DictReader().
  • I can explain the difference between csv.writer() and csv.DictWriter().
Skills
  • I can read a CSV file into a list of dictionaries.
  • I can write a list of dictionaries to a new CSV file, including the header row.
  • I can filter rows based on a numeric or text condition.
  • I can summarize a column: total, average, maximum, and minimum.
  • I can add a new computed column to every row in a table.
  • I can clean messy data: strip whitespace, normalize case, and handle missing values safely.
  • I can convert string values to int or float correctly, and explain what happens when conversion fails.
Application
  • I can combine loading, cleaning, filtering, summarizing, and saving into one organized program.
  • I can build a complete gradebook-style CSV analyzer from a written set of requirements.
Preview — Chapter 23
Exceptions and Error Handling

Chapter 22 built programs that read and write real data. But these programs have a fragile assumption: the files exist, the data is valid, and nothing unexpected happens. In the real world, files are missing. Users type the wrong input. Networks fail. Numbers appear where letters are expected. Scores of "N/A" appear where integers are expected. Directories do not exist. Disks are full.

Chapter 23 teaches exceptions and error handling. You will learn how Python signals that something unexpected happened, how to read a traceback, and how to write try/except blocks that let your programs recover from errors gracefully. You will also revisit input validation and file operations — both of which become much more robust once you can handle exceptions properly. After Chapter 23, your CSV programs will be able to handle missing files, bad data, and unexpected input without crashing. That is what it means to write programs that are ready for real use.

MGU Library Connection
  • Programming with Python — Appendix I: File and CSV Reference
  • Programming with Python — Appendix J: Standard Library Beginner Map
  • Companion resource: CSV Processing Guide
  • Companion resource: Dictionary Methods Chart
  • Next chapter: Exceptions and Error Handling (Chapter 23)
  • Related course: Python for Data Science — builds on these table-processing foundations