Cohort 4Register

PART 4 · DSA AND COMPLEXITY · CHAPTER 27

Graphs: BFS, cycles and topological sort

FreeLesson 52 of 78

A graph is things connected to things. An Airflow DAG is a graph, and scheduling its tasks in a valid order is a textbook algorithm.

RUN ORDER FOR A PIPELINE DAG, AND CATCHING A CYCLEPYTHON 3.12
from collections import defaultdict, deque

edges = [("extract_orders", "clean_orders"),
         ("extract_customers", "dim_customer"),
         ("clean_orders", "fact_orders"),
         ("dim_customer", "fact_orders"),
         ("fact_orders", "daily_report"),
         ("dim_customer", "daily_report")]

def topo_order(edges):                       # Kahn's algorithm
    graph, indegree = defaultdict(list), defaultdict(int)
    for a, b in edges:
        graph[a].append(b)
        indegree[b] += 1
        indegree.setdefault(a, 0)
    ready = deque(sorted(n for n, d in indegree.items() if d == 0))
    order = []
    while ready:
        n = ready.popleft()
        order.append(n)
        for nxt in graph[n]:
            indegree[nxt] -= 1
            if indegree[nxt] == 0:           # all its inputs are done
                ready.append(nxt)
    if len(order) < len(indegree):
        raise ValueError("cycle: some tasks wait on each other")
    return order

print(topo_order(edges))
try:
    topo_order(edges + [("daily_report", "extract_orders")])
except ValueError as e:
    print(e)

Output

['extract_customers', 'extract_orders', 'dim_customer', 'clean_orders', 'fact_orders', 'daily_report']
cycle: some tasks wait on each other
BFS ON A GRID: COUNT CONNECTED GROUPSPYTHON 3.12
from collections import deque

grid = ["11000",
        "11010",
        "00011",
        "00000"]

def islands(grid):
    rows, cols, seen, count = len(grid), len(grid[0]), set(), 0
    for r in range(rows):
        for c in range(cols):
            if grid[r][c] == "1" and (r, c) not in seen:
                count += 1                       # new island: flood it
                q = deque([(r, c)]); seen.add((r, c))
                while q:
                    y, x = q.popleft()
                    for dy, dx in ((1, 0), (-1, 0), (0, 1), (0, -1)):
                        ny, nx = y + dy, x + dx
                        inside = 0 <= ny < rows and 0 <= nx < cols
                        if (inside and grid[ny][nx] == "1"
                                and (ny, nx) not in seen):
                            seen.add((ny, nx)); q.append((ny, nx))
    return count

print(islands(grid))

Output

2

"Number of islands" is the most asked graph question. A grid is a graph where each cell links to its four neighbours. Every cell is visited once: O(rows x cols).

Mark a cell as seen when you add it to the queue, not when you pop it, or the same cell can be queued many times.

The question asksReach forCost
Run order of dependent tasksTopological sort (Kahn)O(V + E)
Is there a cycle in the dependencies?Kahn leftover count, or DFS with "visiting" marksO(V + E)
Fewest steps between two nodes, equal step costBFSO(V + E)
Cheapest path with different costsDijkstra's algorithm with a heapO(E log V)
Which items are connected (groups, islands)BFS / DFS flood fill, or union-findO(V + E)

Practice before you move on

  1. Add a task that depends on itself to the DAG and check that the cycle check catches it.
  2. Change the grid in the BFS example and count the groups again.