Cohort 4Register

PART 2 · PYTHON FOUNDATIONS · CHAPTER 4

Dicts, defaultdict and Counter

FreeLesson 27 of 78

The dict is the most important data structure in data engineering Python. Grouping, counting, joining and deduping all start with one.

LOOKUPS THAT DO NOT CRASHPYTHON 3.12
import json
customers = json.load(open("customers.json"))
by_id = {c["customer_id"]: c for c in customers}

print(by_id[2]["name"])
print(by_id.get(99))            # None, no KeyError
print(by_id.get(99, {}).get("name", "unknown"))

sana = by_id[6]
print(sana["address"]["city"])  # key exists, is None
city = (sana.get("address") or {}).get("city") or "Unknown"
print(city)

Output

Diya
None
unknown
None
Unknown
GROUP, COUNT, MERGEPYTHON 3.12
import csv
from collections import Counter, defaultdict

rows = list(csv.DictReader(open("order_items.csv")))

items_per_order = defaultdict(list)       # group
for r in rows:
    items_per_order[r["order_id"]].append(r["product_id"])
print(dict(list(items_per_order.items())[:3]))

counts = Counter(r["product_id"] for r in rows)   # count
print(counts.most_common(3))

merged = {"retries": 3, "env": "dev"} | {"env": "prod"}
print(merged)                             # right side wins

Output

{'101': ['1', '5'], '102': ['3', '6'], '103': ['2']}
[('1', 5), ('5', 4), ('3', 3)]
{'retries': 3, 'env': 'prod'}
DELETING WHILE ITERATINGPYTHON 3.12
prices = {"mouse": 799, "hub": 1499, "pens": 149}
for name, p in prices.items():
    if p < 500:
        del prices[name]

Output

RuntimeError: dictionary changed size during iteration
BUILD A NEW DICT INSTEADPYTHON 3.12
prices = {"mouse": 799, "hub": 1499, "pens": 149}
prices = {k: v for k, v in prices.items()
          if v >= 500}
print(prices)

Output

{'mouse': 799, 'hub': 1499}

Practice before you move on

  1. Build a defaultdict(list) that maps each customer_id in orders.csv to its order_ids.
  2. Use Counter to find the two most common statuses, then add two Counters together with +.

Watch the free lesson

The same topic in the Free Lessons library, free with a sign-in.