Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 66 additions & 2 deletions src/content/foundations/big-o-notation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,75 @@ loops multiply; halving the search space each step is logarithmic.
for x in list: # O(n)
do_work(x)

for x in list: # O(n²) — nested over the same input
for x in list: # O(n²) — nested over the same input
for y in list:
compare(x, y)

while lo <= hi: # O(log n) — the range halves each iteration
while lo <= hi: # O(log n) — the range halves each iteration
mid = (lo + hi) / 2
```

**Visualizing nested loops as a matrix.** The reason nested loops multiply their bounds
becomes obvious if you plot `(i, j)` as coordinates on a grid. A double loop where both `i`
and `j` range over the same input of size `n` visits every cell of an `n × n` matrix exactly
once — the outer loop selects a row, the inner loop sweeps that row left to right:

```
j=0 j=1 j=2 j=3 ... j=n-1
i=0 • • • • ... •
i=1 • • • • ... •
i=2 • • • • ... •
i=3 • • • • ... •
...
i=n-1 • • • • ... •
```

Each `•` is one execution of the inner loop body — one `compare(x, y)`. Summing cells row by
row is the formal derivation of the bound, not just an analogy:

```
T(n) = Σ (i=0 to n-1) Σ (j=0 to n-1) 1
= Σ (i=0 to n-1) n # inner sum: n cells per row, independent of i
= n · n
= n²
```

So `O(n²)` isn't a rule to memorize — it's the area of the grid the two indices jointly
traverse. This generalizes directly: an `m × n` matrix from two independent loops of
different lengths gives `T = m · n`, i.e. `O(m · n)`, not `O(n²)` — the matrix is rectangular,
not square, and reading the shape off the loop bounds prevents that misclassification.

**Triangular traversal: when `j` depends on `i`.** A very common pattern is comparing each
element only to the ones after it (`for j in range(i + 1, n)`), used in bubble sort, pairwise
comparisons, and the [dedup example](#example) below if written to avoid double-counting.
Here the inner loop's range shrinks as `i` grows, so only the upper triangle of the matrix is
visited:

```
j=0 j=1 j=2 j=3
i=0 • • •
i=1 • •
i=2 •
i=3
```

Row `i` now contributes `(n - 1 - i)` cells instead of `n`. Summing the arithmetic series:

```
T(n) = Σ (i=0 to n-1) (n - 1 - i)
= (n-1) + (n-2) + ... + 1 + 0
= n(n-1) / 2
```

`n(n-1)/2` is still `O(n²)` — the leading term dominates once lower-order terms and constant
factors are dropped, per the definition in <What>. The triangle is exactly half of the `n × n`
square it's inscribed in, so the constant factor drops (roughly 2×) but the growth **class**
does not change. This is a frequent point of real-world confusion: an engineer sees the
iteration count roughly halve in profiling output and assumes a better complexity class, when
only the constant improved. The matrix view resolves this at a glance — a triangle sits inside
the same `n × n` square it's half of, so no triangular traversal over a single input can be
sub-quadratic.

**The growth classes that matter**, and where you meet them:

| Class | Name | Example | 1M items |
Expand Down Expand Up @@ -164,6 +225,9 @@ disagrees.
with worst-case `O(n)` shows up as p99 latency spikes.
- **Forgetting space.** An algorithm that is fast and allocates a copy per element will
hit memory limits or GC pressure long before it hits a CPU limit.
- **Mistaking a reduced constant for a reduced complexity class.** A triangular traversal
visits roughly half the cells of the equivalent square matrix, but stays `O(n²)` — see the
matrix walkthrough in <How>.

</Pitfalls>

Expand Down