Cohort 4Register

PART 3 · ADVANCED PYTHON · CHAPTER 12

Decorators: timing, retry, context managers

FreeLesson 36 of 78

A decorator wraps a function to add behaviour around it: timing, retries, logging, caching. Pipelines use them everywhere.

A TIMING DECORATORPYTHON 3.12
import functools, time

def timed(fn):
    @functools.wraps(fn)            # keep name and docstring
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = fn(*args, **kwargs)
        took = time.perf_counter() - start
        print(f"{fn.__name__} took {took:.1f}s")
        return result
    return wrapper

@timed
def load_batch(rows):
    "Pretend to write rows to the warehouse."
    time.sleep(0.2)
    return len(rows)

print(load_batch([1, 2, 3]), load_batch.__name__)
print(load_batch.__doc__)

Output

load_batch took 0.2s
3 load_batch
Pretend to write rows to the warehouse.

The times are what this run measured when the book was built. Yours will differ.

A DECORATOR THAT TAKES ARGUMENTSPYTHON 3.12
import functools, time

def retry(times=3, delay=0.01, on=(ConnectionError,)):
    def decorator(fn):
        @functools.wraps(fn)
        def wrapper(*args, **kwargs):
            for attempt in range(1, times + 1):
                try:
                    return fn(*args, **kwargs)
                except on as e:
                    print(f"attempt {attempt} failed: {e}")
                    if attempt == times:
                        raise
                    time.sleep(delay * 2 ** (attempt - 1))
        return wrapper
    return decorator

calls = {"n": 0}
@retry(times=3)
def fetch_page():
    calls["n"] += 1
    if calls["n"] < 3:
        raise ConnectionError("warehouse timeout")
    return "page ok"
print(fetch_page())

Output

attempt 1 failed: warehouse timeout
attempt 2 failed: warehouse timeout
page ok
@CONTEXTMANAGER: SETUP, YIELD, TEARDOWNPYTHON 3.12
from contextlib import contextmanager

@contextmanager
def step(name):
    print(f"start {name}")
    try:
        yield
        print(f"done  {name}")
    except Exception as e:
        print(f"fail  {name}: {e}")
        raise
    finally:
        print(f"cleanup {name}")
with step("extract"):
    rows = [1, 2, 3]
try:
    with step("load"):
        raise TimeoutError("warehouse timeout")
except TimeoutError:
    pass

Output

start extract
done  extract
cleanup extract
start load
fail  load: warehouse timeout
cleanup load

Everything before yield is setup, everything after it is teardown. The finally block runs whether the step passed or failed, which is where you release locks, close connections or delete temp files.

Practice before you move on

  1. Put the timing decorator on a function that reads orders.csv and print how long it took.
  2. Change the retry decorator so it retries only on ValueError, and test it with a function that fails twice.