Tony CantaveCounting Things Fast with Python's Counter and defaultdict
By Tony Cantave ·
A lot of the questions I get asked about data are counting questions. How many stops went to each town last week? Which part numbers show up most often on the scrap report? How many rows in this export are missing an email address? In my years running delivery and logistics operations, and later managing production with ERP and SAP exports, I answered those questions in spreadsheets with pivot tables. Today I answer most of them with a few lines of Python, and two tools from the standard library do almost all of the work: Counter and defaultdict.
This post walks through how I use both, with code you can paste into a script and run.
The problem with counting by hand in a dict
The first version everyone writes looks like this:
counts = {}
for town in towns:
if town in counts:
counts[town] += 1
else:
counts[town] = 1It works, but the if and else are noise, and the noise grows once you start grouping things into lists instead of counting them. The collections module removes it.
Counter: tallies in one line
from collections import Counter
towns = ["New Haven", "Bridgeport", "New Haven", "Hamden", "New Haven", "Bridgeport"]
counts = Counter(towns)
print(counts)
# Counter({'New Haven': 3, 'Bridgeport': 2, 'Hamden': 1})
print(counts.most_common(2))
# [('New Haven', 3), ('Bridgeport', 2)]most_common is the method I use most. It returns pairs sorted from highest to lowest, which is exactly the shape you want for a quick report.
A few other things Counter does that save time:
- Missing keys return 0 instead of raising KeyError, so counts["Stamford"] is simply 0.
- You can keep adding as you go with counts.update(more_towns).
- Two counters can be added together, which is handy when you process one file per day and want a weekly total: week = monday + tuesday + wednesday.
- counts.total() (Python 3.10 and newer) or sum(counts.values()) gives you the grand total.
Counting from a CSV file
Most of my real input is a CSV export. Here is the pattern I reach for:
import csv
from collections import Counter
def count_column(path, column):
counts = Counter()
with open(path, newline="", encoding="utf8") as f:
for row in csv.DictReader(f):
value = (row.get(column) or "").strip()
counts[value or "(blank)"] += 1
return counts
if __name__ == "__main__":
result = count_column("stops.csv", "town")
for town, n in result.most_common():
print(f"{town:20} {n}")Two details matter here. First, I strip whitespace, because "New Haven" and "New Haven " will otherwise be counted as two different towns. Second, I label empty values as "(blank)" so they show up in the report instead of hiding. The count of blanks is often the most useful number in the whole output, because it tells you how much of the data you cannot trust yet.
If capitalization is inconsistent, normalize it too, for example with value.title() for town names or value.upper() for part numbers.
defaultdict: grouping instead of counting
Sometimes I do not want a number. I want the actual rows grouped by a key. That is where defaultdict comes in.
import csv
from collections import defaultdict
def group_rows(path, column):
groups = defaultdict(list)
with open(path, newline="", encoding="utf8") as f:
for row in csv.DictReader(f):
key = (row.get(column) or "").strip() or "(blank)"
groups[key].append(row)
return groups
groups = group_rows("stops.csv", "town")
for town, rows in sorted(groups.items()):
print(town, len(rows))defaultdict(list) creates an empty list the first time you touch a new key, so append always works. Other factories are just as useful:
- defaultdict(int) behaves like a simple counter.
- defaultdict(set) collects unique values, for example every driver who served a given town, or every customer who ordered a given part.
- defaultdict(float) is good for summing amounts, like total weight or total hours per category.
Here is the sum pattern, which replaces a lot of pivot tables:
from collections import defaultdict
totals = defaultdict(float)
for row in rows:
totals[row["category"]] += float(row["hours"] or 0)Picking the right one
My rule of thumb is simple. If the answer to the question is a number per key, use Counter. If the answer is a list, a set, or a sum per key, use defaultdict with the matching factory. If you need both, group first with defaultdict and then count with len(), or build a Counter from one field of each group.
Mistakes I made early on
- Counting before cleaning. Normalize whitespace and case first, or your top ten list will contain duplicates.
- Forgetting that defaultdict creates keys when you read them. Looking up groups["Stamford"] quietly adds an empty Stamford entry. Use "Stamford" in groups or groups.get("Stamford") when you only want to look.
- Printing a giant dict. Use most_common(10) or sort the items so the output reads like a report, not a dump.
- Converting numbers too late. Turn strings into int or float as you read each row, and decide up front what a blank should mean.
Why this matters
None of this is fancy, and that is the point. When I moved from logistics and manufacturing into software and AI evaluation work, the skills that paid off first were the small, boring ones: reading a file correctly, cleaning values, and summarizing them in a way someone else can check. Counter and defaultdict are two of the smallest tools in Python, and they answer a surprising share of everyday data questions in under twenty lines.
If you have a spreadsheet you pivot by hand every week, try rewriting that one report as a script. It is a good first automation project, and you will reuse the pattern constantly.