business_cycle.py validate() rejects the committed snapshots under pandas 3: stack() no longer drops NaN

[quantecon-services]

Found during the #127 validation (section 3, the builder-layer margin). Not introduced by #126 — the line dates from #114 (ebb1bd1, 2026-09-01) — but #126 kept it, and it is the one place where the two-layer validation is not green on both pandas majors.

What happens

Under pandas 3.0.5 (numpy 2.4.6), the World Bank builder's own validate() fails on the committed, valid bytes of all three tables it writes:

gdp_growth_annual.csv: value out of band [-50, 50]
unemployment_rate_annual.csv: value out of band [0, 60]
private_credit_to_gdp.csv: value out of band [0, 400]

Reproduce (no network; wbgapi stubbed so the module imports):

import sys, types; sys.path.insert(0, 'builders'); sys.modules['wbgapi'] = types.ModuleType('wbgapi')
import pandas as pd, business_cycle as bc
for t in bc.TABLES:
    bc.validate(t, pd.read_csv('lectures/' + t['file'], index_col=0))   # raises under pandas 3, passes under 2.3.3

The shared builders/_validate.py passes the same frames under both versions (44/44 via scripts/validate_datasets.py on 2.3.3 and 3.0.5), and business_cycle_fred.py's validate() passes under both. Only this builder is affected.

Cause

builders/business_cycle.py:144:

_check(values.stack().between(lo, hi).all(), f'{name}: value out of band [{lo}, {hi}]')

pandas 3 made the new DataFrame.stack() implementation the default, and it keeps NaN (there is no dropna= any more). The structural nulls the manifests declare — every economy's YR1960, the leading nulls before GBR's and FRA's first observations, an unpublished newest year — become NaN rows in the stacked series, between() is False for each of them, and .all() fails. Under 2.3.3 stack() dropped them.

Measured:

pandas 3.0.5 stack len 4 has NaN True -> between.all() = False
pandas 2.3.3 stack len 2 has NaN False -> between.all() = True

Why it matters

requirements.txt pins pandas 2.3.3, so the canary and the refresh job are green today. But the lectures run pandas 3 (anaconda=2026.07), requirements.txt says the pin "records the measurement rather than guarding a known sensitivity", and validate-datasets.yml describes running 2.3.3 as "the standing check that it stays so [green on both]". This builder is a known sensitivity: the first time the pin moves (or someone dry-runs the builder in the lectures' environment) every World Bank canary leg goes red with a message that points at the data rather than at pandas.

Fix

Drop the nulls before the band check, since the placement rule in the manifest already governs where they may sit:

_check(values.stack().dropna().between(lo, hi).all(), f'{name}: value out of band [{lo}, {hi}]')

builders/_template.py does not carry the idiom (it leaves bands to the copying builder), so a one-line note there — "use .dropna() before a band check; pandas 3 stack() keeps NaN" — would stop the next builder inheriting it.