Refresh business_cycle_data.csv: 2023 → 2025

AGENTS.md

@@ -120,7 +120,7 @@ Where one builder produces a **set** of files, name it for the set and let each
 
 Builders follow four stages — **fetch → pre-process → validate → write** — and only write on validation pass (expected columns/dtypes, row-count floor, recency of date range, no all-NaN columns, and a **bounded** overlap window against the previous vintage — a tracking snapshot is revised by its source, so the test is a tolerance plus a printed summary, never equality). Lectures always read the last-good snapshot: an upstream outage may fail a refresh, it must never break a lecture build.
 
-**A dynamic snapshot's builder also honours the refresh contract** that `.github/workflows/refresh-snapshots.yml` and `scripts/snapshots.py` rely on — copy `builders/_template.py`: `--out-dir` (dry run for the weekly canary), `--summary-json` (the run summary the manifest stamp and the refresh PR body are built from), writes through a temp file and `os.replace()`, and exit code **2** for a `ValidationError` against **1** for a fetch failure, which is how the canary issue tells "the data broke the contract" from "the network was down". The manifest fields the workflow stamps (`retrieved`, `integrity.sha256`, `integrity.upstream.*`, `schema.date_range.end`) must be single-line values with their reasoning in comments **above** them, not beside — the stamp replaces the line.
+**A dynamic snapshot's builder also honours the refresh contract** that `.github/workflows/refresh-snapshots.yml` and `scripts/snapshots.py` rely on — copy `builders/_template.py`: `--out-dir` (dry run for the weekly canary), `--summary-json` (the run summary the manifest stamp and the refresh PR body are built from), writes through a temp file and `os.replace()`, and exit code **2** for a `ValidationError` against **1** for a fetch failure, which is how the canary issue tells "the data broke the contract" from "the network was down". The manifest fields the workflow stamps (`retrieved`, `integrity.sha256`, `integrity.upstream.*`, `schema.date_range.end`) must be single-line values with their reasoning in comments **above** them, not beside — the stamp replaces the line. And **no prose in the manifest may embed a fact a refresh can change** — an end year, an observed range, a row count — because nothing re-writes prose: the first refresh PR (#112) shipped a column description still saying "YR2023 in the committed bytes".
 
 ### Live APIs
 

CATALOG.md

@@ -18,7 +18,7 @@ The dataset registry, **auto-generated** from the sidecar manifests (`lectures/*
 | [**assignat.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/assignat.xlsx)<br><sub>French Revolution — assignat issues, budgets and seigniorage (Sargent-Velde)</sub> | verbatim | [Sargent and Velde, "Macroeconomic Features of the French Revolution" — supporting spreadsheets](https://www.journals.uchicago.edu/doi/10.1086/261992) |  | ✅ permitted | ⚠️ unverifiable | n/a (verbatim) | 204.6 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)<br>[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)<br>[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)<br>[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md)<br>[tom-econ370-2025 · french_rev.md](https://github.com/QuantEcon/tom-econ370-2025/blob/main/lectures/french_rev.md)<br><sub>⚠️ BROKEN reader (measured 2026-08-19): fetches this dataset through a stale `base_url` still pointing at lecture-python-intro's deleted `datasets/` copy (french_rev.md:70-75), which serves 404</sub> |
 | [**bbh_macro_quarterly.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/bbh_macro_quarterly.csv)<br><sub>Bhandari-Borovička-Ho replication — quarterly US macro series for the belief-wedge VAR, 1955Q1-2019Q4</sub> | constructed | [Replication package for "Survey data and subjective beliefs in business cycle models" (Bhandari, Borovička and Ho), file `data input/FRED/data_FRED.xlsx`](https://doi.org/10.5281/zenodo.10194324) | CC-BY-4.0 | ✅ permitted | ✅ verified | ✅ committed | 31.5 KB | [lecture-python-advanced.myst · subjective_beliefs_business_cycles.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/subjective_beliefs_business_cycles.md) |
 | [**bbh_michigan_monthly.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/bbh_michigan_monthly.csv)<br><sub>Michigan Surveys of Consumers monthly aggregates and the US unemployment rate, 1978-01 to 2020-03 (BBH replication extract)</sub> | constructed | [Bhandari, Borovička and Ho replication package (Zenodo), carrying University of Michigan Surveys of Consumers published aggregates and a US Bureau of Labor Statistics series retrieved via FRED](https://doi.org/10.5281/zenodo.10194324) | CC-BY-4.0 | ⚠️ restricted | ✅ verified | ✅ committed | 11.9 KB | [lecture-python-advanced.myst · subjective_beliefs_business_cycles.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/subjective_beliefs_business_cycles.md) |
-| [**business_cycle_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/business_cycle_data.csv)<br><sub>World Bank GDP growth (annual %) — USA, ARG, GBR, GRC, JPN, 1960 onward</sub> | dynamic-snapshot | [World Bank, World Development Indicators (national accounts data, and OECD National Accounts data files)](https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG) | CC BY-4.0 | ✅ permitted | ⇄ diverged | ✅ committed | 5.8 KB | — |
+| [**business_cycle_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/business_cycle_data.csv)<br><sub>World Bank GDP growth (annual %) — USA, ARG, GBR, GRC, JPN, 1960 onward</sub> | dynamic-snapshot | [World Bank, World Development Indicators (national accounts data, and OECD National Accounts data files)](https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG) | CC BY-4.0 | ✅ permitted | ✅ verified | ✅ committed | 6.0 KB | — |
 | [**caron.npy**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/caron.npy)<br><sub>French Revolution — monthly specie value of the assignat, 1791-1796</sub> | constructed | unrecorded |  | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 1.1 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)<br>[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)<br>[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)<br>[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md)<br><sub>⚠️ Reads a local `datasets/` copy, not this file</sub><br>[tom-econ370-2025 · french_rev.md](https://github.com/QuantEcon/tom-econ370-2025/blob/main/lectures/french_rev.md)<br><sub>⚠️ Course fork with a live Pages site; reads its own blob-identical `datasets/` copy (french_rev.md:715-716), not this file, and its `base_url` still points at lecture-python-intro</sub><br>[python-lecture-sandpit.myst · french_rev.md](https://github.com/QuantEcon/python-lecture-sandpit.myst/blob/main/lectures/french_rev.md)<br><sub>⚠️ Public sandpit holding `lectures/_static/` copies</sub> |
 | [**chapter_3.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/chapter_3.xlsx)<br><sub>The Ends of Four Big Inflations — appendix tables, transcribed</sub> | constructed | [Sargent, "Rational Expectations and Inflation", chapter 3 appendix tables](https://press.princeton.edu/books/paperback/9780691158709/rational-expectations-and-inflation) |  | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 71.6 KB | [lecture-python-intro · inflation_history.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/inflation_history.md)<br>[lecture-wasm · inflation_history.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/inflation_history.md)<br>[lecture-intro.zh-cn · inflation_history.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/inflation_history.md)<br>[test-actions-lecture-intro · inflation_history.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/inflation_history.md) |
 | [**cities_brazil.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/cities_brazil.csv)<br><sub>World Population Review — Brazilian city populations, 2023</sub> | verbatim | [World Population Review — cities in Brazil](https://worldpopulationreview.com/countries/cities/brazil) |  | ⚠️ restricted | ⚠️ unverifiable | n/a (verbatim) | 17.5 KB | [lecture-python-intro · heavy_tails.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/heavy_tails.md)<br>[lecture-wasm · heavy_tails.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/heavy_tails.md)<br>[lecture-intro.zh-cn · heavy_tails.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/heavy_tails.md)<br>[test-actions-lecture-intro · heavy_tails.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/heavy_tails.md) |

builders/_template.py

@@ -26,7 +26,11 @@
               equality. Measure the source's routine revisions before
               choosing the bound (business_cycle: observed max 1.5 pp,
               bound 5 pp)
-  provenance  upstream metadata dumps go to provenance/, never lectures/
+  provenance  upstream metadata dumps go to provenance/, never lectures/;
+              collapse runs of blank lines so a refresh diff shows content
+  manifest    prose in the sidecar must not embed facts a refresh can change
+              (an end year, an observed range, a row count) -- those live in
+              the fields scripts/snapshots.py stamps, and only there
 
 Requires pandas plus whatever the source needs (add it to requirements.txt).
 """

builders/business_cycle.py

@@ -29,7 +29,10 @@
 Two provenance dumps are written beside the data, to provenance/ (NOT
 lectures/ -- they are not datasets; QuantEcon/data-lectures#13): the series
 metadata, which is where the CC BY-4.0 licence the manifest cites is stated,
-and the `wb.series.info(q='GDP growth')` listing the lecture teaches.
+and the `wb.series.info(q='GDP growth')` listing the lecture teaches. Runs of
+blank lines in the World Bank's text are collapsed to one (whitespace carries
+no evidence, and the raw dump had runs ten newlines deep), so a refresh diff
+of the dump shows what the source SAID, not how it was padded.
 
 Stages: fetch -> pre-process -> validate -> write. Writes only on validation
 pass, and each write goes through a temp file and os.replace(), so neither a
@@ -103,11 +106,16 @@ def _check(condition, message):
         raise ValidationError(message)
 
 
+def _tidy(text):
+    """Collapse runs of blank lines in an upstream text dump to one."""
+    return re.sub(r'\n{3,}', '\n\n', text)
+
+
 def fetch():
     """Live WDI, exactly the lecture's own three calls."""
     frame = wb.data.DataFrame(SERIES, ECONOMIES, labels=True)
-    metadata = str(wb.series.metadata.get(SERIES))
-    info = str(wb.series.info(q='GDP growth'))
+    metadata = _tidy(str(wb.series.metadata.get(SERIES)))
+    info = _tidy(str(wb.series.info(q='GDP growth')))
     return frame, metadata, info