diff --git a/AGENTS.md b/AGENTS.md index 46f98db..ea959de 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -120,7 +120,7 @@ Where one builder produces a **set** of files, name it for the set and let each Builders follow four stages — **fetch → pre-process → validate → write** — and only write on validation pass (expected columns/dtypes, row-count floor, recency of date range, no all-NaN columns, and a **bounded** overlap window against the previous vintage — a tracking snapshot is revised by its source, so the test is a tolerance plus a printed summary, never equality). Lectures always read the last-good snapshot: an upstream outage may fail a refresh, it must never break a lecture build. -**A dynamic snapshot's builder also honours the refresh contract** that `.github/workflows/refresh-snapshots.yml` and `scripts/snapshots.py` rely on — copy `builders/_template.py`: `--out-dir` (dry run for the weekly canary), `--summary-json` (the run summary the manifest stamp and the refresh PR body are built from), writes through a temp file and `os.replace()`, and exit code **2** for a `ValidationError` against **1** for a fetch failure, which is how the canary issue tells "the data broke the contract" from "the network was down". The manifest fields the workflow stamps (`retrieved`, `integrity.sha256`, `integrity.upstream.*`, `schema.date_range.end`) must be single-line values with their reasoning in comments **above** them, not beside — the stamp replaces the line. +**A dynamic snapshot's builder also honours the refresh contract** that `.github/workflows/refresh-snapshots.yml` and `scripts/snapshots.py` rely on — copy `builders/_template.py`: `--out-dir` (dry run for the weekly canary), `--summary-json` (the run summary the manifest stamp and the refresh PR body are built from), writes through a temp file and `os.replace()`, and exit code **2** for a `ValidationError` against **1** for a fetch failure, which is how the canary issue tells "the data broke the contract" from "the network was down". The manifest fields the workflow stamps (`retrieved`, `integrity.sha256`, `integrity.upstream.*`, `schema.date_range.end`) must be single-line values with their reasoning in comments **above** them, not beside — the stamp replaces the line. And **no prose in the manifest may embed a fact a refresh can change** — an end year, an observed range, a row count — because nothing re-writes prose: the first refresh PR (#112) shipped a column description still saying "YR2023 in the committed bytes". ### Live APIs diff --git a/CATALOG.md b/CATALOG.md index 0ec2ca6..e8f6485 100644 --- a/CATALOG.md +++ b/CATALOG.md @@ -18,7 +18,7 @@ The dataset registry, **auto-generated** from the sidecar manifests (`lectures/* | [**assignat.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/assignat.xlsx)
French Revolution — assignat issues, budgets and seigniorage (Sargent-Velde) | verbatim | [Sargent and Velde, "Macroeconomic Features of the French Revolution" — supporting spreadsheets](https://www.journals.uchicago.edu/doi/10.1086/261992) | | ✅ permitted | ⚠️ unverifiable | n/a (verbatim) | 204.6 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)
[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)
[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)
[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md)
[tom-econ370-2025 · french_rev.md](https://github.com/QuantEcon/tom-econ370-2025/blob/main/lectures/french_rev.md)
⚠️ BROKEN reader (measured 2026-08-19): fetches this dataset through a stale `base_url` still pointing at lecture-python-intro's deleted `datasets/` copy (french_rev.md:70-75), which serves 404 | | [**bbh_macro_quarterly.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/bbh_macro_quarterly.csv)
Bhandari-Borovička-Ho replication — quarterly US macro series for the belief-wedge VAR, 1955Q1-2019Q4 | constructed | [Replication package for "Survey data and subjective beliefs in business cycle models" (Bhandari, Borovička and Ho), file `data input/FRED/data_FRED.xlsx`](https://doi.org/10.5281/zenodo.10194324) | CC-BY-4.0 | ✅ permitted | ✅ verified | ✅ committed | 31.5 KB | [lecture-python-advanced.myst · subjective_beliefs_business_cycles.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/subjective_beliefs_business_cycles.md) | | [**bbh_michigan_monthly.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/bbh_michigan_monthly.csv)
Michigan Surveys of Consumers monthly aggregates and the US unemployment rate, 1978-01 to 2020-03 (BBH replication extract) | constructed | [Bhandari, Borovička and Ho replication package (Zenodo), carrying University of Michigan Surveys of Consumers published aggregates and a US Bureau of Labor Statistics series retrieved via FRED](https://doi.org/10.5281/zenodo.10194324) | CC-BY-4.0 | ⚠️ restricted | ✅ verified | ✅ committed | 11.9 KB | [lecture-python-advanced.myst · subjective_beliefs_business_cycles.md](https://github.com/QuantEcon/lecture-python-advanced.myst/blob/main/lectures/subjective_beliefs_business_cycles.md) | -| [**business_cycle_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/business_cycle_data.csv)
World Bank GDP growth (annual %) — USA, ARG, GBR, GRC, JPN, 1960 onward | dynamic-snapshot | [World Bank, World Development Indicators (national accounts data, and OECD National Accounts data files)](https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG) | CC BY-4.0 | ✅ permitted | ⇄ diverged | ✅ committed | 5.8 KB | — | +| [**business_cycle_data.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/business_cycle_data.csv)
World Bank GDP growth (annual %) — USA, ARG, GBR, GRC, JPN, 1960 onward | dynamic-snapshot | [World Bank, World Development Indicators (national accounts data, and OECD National Accounts data files)](https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG) | CC BY-4.0 | ✅ permitted | ✅ verified | ✅ committed | 6.0 KB | — | | [**caron.npy**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/caron.npy)
French Revolution — monthly specie value of the assignat, 1791-1796 | constructed | unrecorded | | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 1.1 KB | [lecture-python-intro · french_rev.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/french_rev.md)
[lecture-wasm · french_rev.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/french_rev.md)
[lecture-intro.zh-cn · french_rev.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/french_rev.md)
[test-actions-lecture-intro · french_rev.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/french_rev.md)
⚠️ Reads a local `datasets/` copy, not this file
[tom-econ370-2025 · french_rev.md](https://github.com/QuantEcon/tom-econ370-2025/blob/main/lectures/french_rev.md)
⚠️ Course fork with a live Pages site; reads its own blob-identical `datasets/` copy (french_rev.md:715-716), not this file, and its `base_url` still points at lecture-python-intro
[python-lecture-sandpit.myst · french_rev.md](https://github.com/QuantEcon/python-lecture-sandpit.myst/blob/main/lectures/french_rev.md)
⚠️ Public sandpit holding `lectures/_static/` copies | | [**chapter_3.xlsx**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/chapter_3.xlsx)
The Ends of Four Big Inflations — appendix tables, transcribed | constructed | [Sargent, "Rational Expectations and Inflation", chapter 3 appendix tables](https://press.princeton.edu/books/paperback/9780691158709/rational-expectations-and-inflation) | | ✅ permitted | ⚠️ unverifiable | ⚠️ unrecovered | 71.6 KB | [lecture-python-intro · inflation_history.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/inflation_history.md)
[lecture-wasm · inflation_history.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/inflation_history.md)
[lecture-intro.zh-cn · inflation_history.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/inflation_history.md)
[test-actions-lecture-intro · inflation_history.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/inflation_history.md) | | [**cities_brazil.csv**](https://github.com/QuantEcon/data-lectures/raw/main/lectures/cities_brazil.csv)
World Population Review — Brazilian city populations, 2023 | verbatim | [World Population Review — cities in Brazil](https://worldpopulationreview.com/countries/cities/brazil) | | ⚠️ restricted | ⚠️ unverifiable | n/a (verbatim) | 17.5 KB | [lecture-python-intro · heavy_tails.md](https://github.com/QuantEcon/lecture-python-intro/blob/main/lectures/heavy_tails.md)
[lecture-wasm · heavy_tails.md](https://github.com/QuantEcon/lecture-wasm/blob/main/lectures/heavy_tails.md)
[lecture-intro.zh-cn · heavy_tails.md](https://github.com/QuantEcon/lecture-intro.zh-cn/blob/main/lectures/heavy_tails.md)
[test-actions-lecture-intro · heavy_tails.md](https://github.com/QuantEcon/test-actions-lecture-intro/blob/main/lectures/heavy_tails.md) | diff --git a/builders/_template.py b/builders/_template.py index c4af91c..6fb70e9 100644 --- a/builders/_template.py +++ b/builders/_template.py @@ -26,7 +26,11 @@ equality. Measure the source's routine revisions before choosing the bound (business_cycle: observed max 1.5 pp, bound 5 pp) - provenance upstream metadata dumps go to provenance/, never lectures/ + provenance upstream metadata dumps go to provenance/, never lectures/; + collapse runs of blank lines so a refresh diff shows content + manifest prose in the sidecar must not embed facts a refresh can change + (an end year, an observed range, a row count) -- those live in + the fields scripts/snapshots.py stamps, and only there Requires pandas plus whatever the source needs (add it to requirements.txt). """ diff --git a/builders/business_cycle.py b/builders/business_cycle.py index 78075b2..e2b7790 100644 --- a/builders/business_cycle.py +++ b/builders/business_cycle.py @@ -29,7 +29,10 @@ Two provenance dumps are written beside the data, to provenance/ (NOT lectures/ -- they are not datasets; QuantEcon/data-lectures#13): the series metadata, which is where the CC BY-4.0 licence the manifest cites is stated, -and the `wb.series.info(q='GDP growth')` listing the lecture teaches. +and the `wb.series.info(q='GDP growth')` listing the lecture teaches. Runs of +blank lines in the World Bank's text are collapsed to one (whitespace carries +no evidence, and the raw dump had runs ten newlines deep), so a refresh diff +of the dump shows what the source SAID, not how it was padded. Stages: fetch -> pre-process -> validate -> write. Writes only on validation pass, and each write goes through a temp file and os.replace(), so neither a @@ -103,11 +106,16 @@ def _check(condition, message): raise ValidationError(message) +def _tidy(text): + """Collapse runs of blank lines in an upstream text dump to one.""" + return re.sub(r'\n{3,}', '\n\n', text) + + def fetch(): """Live WDI, exactly the lecture's own three calls.""" frame = wb.data.DataFrame(SERIES, ECONOMIES, labels=True) - metadata = str(wb.series.metadata.get(SERIES)) - info = str(wb.series.info(q='GDP growth')) + metadata = _tidy(str(wb.series.metadata.get(SERIES))) + info = _tidy(str(wb.series.info(q='GDP growth'))) return frame, metadata, info diff --git a/lectures/business_cycle_data.csv b/lectures/business_cycle_data.csv index 2e414ff..f2ba36c 100644 --- a/lectures/business_cycle_data.csv +++ b/lectures/business_cycle_data.csv @@ -1,6 +1,6 @@ -economy,Country,YR1960,YR1961,YR1962,YR1963,YR1964,YR1965,YR1966,YR1967,YR1968,YR1969,YR1970,YR1971,YR1972,YR1973,YR1974,YR1975,YR1976,YR1977,YR1978,YR1979,YR1980,YR1981,YR1982,YR1983,YR1984,YR1985,YR1986,YR1987,YR1988,YR1989,YR1990,YR1991,YR1992,YR1993,YR1994,YR1995,YR1996,YR1997,YR1998,YR1999,YR2000,YR2001,YR2002,YR2003,YR2004,YR2005,YR2006,YR2007,YR2008,YR2009,YR2010,YR2011,YR2012,YR2013,YR2014,YR2015,YR2016,YR2017,YR2018,YR2019,YR2020,YR2021,YR2022,YR2023 -JPN,Japan,,12.0435364080117,8.90897299558067,8.47364238279388,11.6767081963981,5.81970787298272,10.6385615591626,11.0821423246927,12.8824681383236,12.4778945144972,0.399050644662793,4.6989920423419,8.413547255176,8.03259997495039,-1.22523982778432,3.09157591622022,3.97498409121211,4.39033795032562,5.27194150290775,5.48404183248661,2.817591207616,4.26062449346863,3.27974274286728,3.63019857224671,4.41088015361535,5.15980790471274,3.29404761337059,4.64886139754491,6.66163652201594,4.92578541306383,4.84092905745234,3.52335723533228,0.900586066360461,-0.459219716150798,1.08338317429946,2.63099961644897,3.13387099280445,0.981228732132223,-1.27033049463975,-0.333929957747415,2.76464755143648,0.386103426145752,0.0419624993125609,1.53512549928013,2.1861156944216,1.80390087227372,1.37235012762038,1.48396941163492,-1.22428900072828,-5.69323635882266,4.09791791930137,0.0238095237748297,1.37475099920874,2.0051001767726,0.296205514142628,1.56062669674904,0.753826745903538,1.67533175166503,0.643391023460339,-0.402169200910933,-4.14711889947692,2.55932023865617,0.954736937781391,1.67901952310368 -GRC,Greece,,13.2038411488938,0.364810821762347,11.8448662648483,9.40967729459825,10.7680111912945,6.4945011330868,5.66948538110917,7.20371933494901,11.5636680387996,8.93058000961291,7.84117728628885,10.1601513015239,8.09237859641172,-6.43824056868939,6.36680845579201,6.85189826017303,2.94100129208149,7.24686326066039,3.28208081242751,0.677130740571783,-1.55372098313507,-1.13264795844424,-1.07862225289738,2.01058079047833,2.50955628089555,0.517660255309238,-2.25886363190979,4.28786188832922,3.79999979077247,0.0,3.10000006906874,0.699999922754131,-1.60000027696664,2.00000001379756,2.09972031337796,1.38247017206857,3.60288320042281,3.43693572579373,2.65878589017217,4.13782709945785,4.65041412229992,4.68313735902784,5.79682303463183,5.37786629728755,1.18350610206699,6.44339775049335,3.50687018582884,0.0574735793110506,-4.11927598076629,-5.69374120273665,-9.87677903873015,-8.33113395866982,-2.27211692215486,0.792225038353592,-0.228301959026894,-0.0317952126675323,1.47312498503697,2.06467253338354,2.27718063644569,-9.19623147082588,8.65449785311438,5.74364918958959,2.3321238825698 -GBR,United Kingdom,,2.70131412786594,1.09869614695408,4.85954532830446,5.59481107154178,2.13033296985384,1.56744975796417,2.77573770930934,5.47269288959338,1.93913785964371,2.70858146236598,3.50471745311494,4.32166756962231,6.52384848717706,-2.48440405800613,-1.47364945113077,2.91026615928646,2.45775049016747,4.20426047029908,3.74901664423379,-2.03136780182236,-0.787743787095451,1.99489123324395,4.22185633739343,2.26910514207148,4.14741495672915,3.15034059663934,5.39273835278517,5.73241357711933,2.57760280436703,0.733755483704996,-1.10312161180532,0.401082055677989,2.48983096244002,3.84600913061144,2.53167001259618,2.58199391653433,4.92464831855712,3.40265821029952,3.05832325913262,4.34163417914475,2.57278038614078,1.79566077025204,3.15234653827419,2.45790441131264,2.73266076130363,2.38069402487066,2.62488001971251,-0.248797298132246,-4.6205537195686,2.2333151263108,1.13836234744893,1.50899837441014,1.79992149362663,3.1946373128021,2.22288844908437,1.92171007763831,2.65650489309972,1.40519026609273,1.62447515947532,-10.2969188737567,8.5759509048565,4.83908514714403,0.339966158003463 -ARG,Argentina,,5.42784287621933,-0.852021521903339,-5.30819682539126,10.1302976615431,10.5694333376394,-0.659726170629824,3.19199662339177,4.82250077932424,9.67952601078326,3.04564331917115,5.65813108184925,1.62841651825939,2.81175403849035,5.5338043957736,-0.0284121037110907,-2.01825151683394,6.93414775653342,-4.50612480201359,10.2227634638681,1.51878394990179,-5.18978914135593,-0.735659159103619,4.34909327926756,1.57073870182023,-5.18902435343904,6.15337706275693,2.70486882770558,-1.08993334133791,-7.15694965597953,-2.46721377810472,9.13311056738961,7.93729155643076,8.20697907221228,5.83620070368526,-2.84520961057079,5.52668982715234,8.1110467707457,3.85017885156228,-3.38545704063269,-0.78899893905691,-4.40883968258557,-10.8944848285903,8.83704079576924,9.02957330068152,8.85165992013437,8.04715150043027,9.00765087504757,4.05723310346406,-5.91852507634947,10.1253981561002,6.00395169280579,-1.0264204544321,2.40532378079436,-2.51261532081394,2.73115982828944,-2.08032784377811,2.81850297775918,-2.61739646282038,-2.00086100285785,-9.9004848136464,10.4418119882506,5.26987967384072,-1.61100162090189 -USA,United States,,2.29999999999953,6.10000000000016,4.39999999999989,5.80000000000027,6.39999999999989,6.50000000000026,2.49999999999999,4.79999999999954,3.09999999999999,0.21660649819502,3.29272245508689,5.25550195050253,5.64567976588816,-0.540550186726477,-0.205619192613284,5.38803389552226,4.62418651704209,5.53520647829761,3.16598822881279,-0.256775732470302,2.53770126730302,-1.80301378770601,4.58379098591793,7.23645315874113,4.1695753468006,3.46265505318908,3.45462966457823,4.17698239778066,3.67223783573645,1.8859655853434,-0.108312888962658,3.52249718409405,2.75179586183873,4.02902272904724,2.68443073540719,3.7727726858688,4.44712794186451,4.48313334577439,4.78842505322162,4.07758575804293,0.955538346430316,1.70044732363073,2.795605965883,3.84777169205226,3.483549937943,2.78453963938148,2.00385829825838,0.113587248258867,-2.57650023426996,2.6951925838264,1.56440685438301,2.28911338767897,2.11783009919849,2.52381981441982,2.94555045455233,1.81945147474293,2.45762230322211,2.96650506916633,2.58382533018855,-2.16302913866514,6.05505293304576,2.51237531983308,2.88755600906016 +economy,Country,YR1960,YR1961,YR1962,YR1963,YR1964,YR1965,YR1966,YR1967,YR1968,YR1969,YR1970,YR1971,YR1972,YR1973,YR1974,YR1975,YR1976,YR1977,YR1978,YR1979,YR1980,YR1981,YR1982,YR1983,YR1984,YR1985,YR1986,YR1987,YR1988,YR1989,YR1990,YR1991,YR1992,YR1993,YR1994,YR1995,YR1996,YR1997,YR1998,YR1999,YR2000,YR2001,YR2002,YR2003,YR2004,YR2005,YR2006,YR2007,YR2008,YR2009,YR2010,YR2011,YR2012,YR2013,YR2014,YR2015,YR2016,YR2017,YR2018,YR2019,YR2020,YR2021,YR2022,YR2023,YR2024,YR2025 +JPN,Japan,,12.043536357022,8.90897303614973,8.47364243635415,11.676708250654,5.81970783966972,10.6385615902031,11.0821423327248,12.8824681707163,12.4778945420024,0.399050611065093,4.69899204214205,8.41354725497627,8.03259997458245,-1.22523982750916,3.09157591623077,3.97498409113672,4.39033795009327,5.27194150317187,5.4840418324284,2.81759120738283,4.20933646523171,3.31245673740017,3.52304498510377,4.50199481622435,5.23338094687848,3.32652561623384,4.73066586040017,6.78502010955764,4.85803768615389,4.89271306574565,3.41749676183089,0.848069581218368,-0.517919847117952,0.99306636322622,2.34205546196849,3.07546731409349,1.36297109407448,-1.77264091037956,-0.347600398675738,2.97832688768516,0.284608556379041,0.0849888050363319,1.38010935830795,2.11294823723127,1.8482813374834,1.54630843078741,1.72837912865577,-1.16662209355141,-5.9359411383964,4.13645589222239,-0.20035785332982,1.64893452981225,1.99727653399265,0.905915486065865,1.79997735783475,0.704986495111257,1.62345062245441,0.834143845965585,-0.308496921143259,-4.28327942675573,3.56419531474475,1.33164750653741,0.720785304749811,-0.240084236041696,1.19311288444599 +GRC,Greece,,13.2038386052462,0.364812289877747,11.8448666655957,9.40967732368279,10.7680109720411,6.49450138060494,5.66948643356486,7.20371768036424,11.5636684082563,8.93058020025967,7.84117657760304,10.1601515908652,8.09237907118801,-6.43824057014004,6.36680794602181,6.85189869571943,2.94100090620763,7.24686397906468,3.28207958853159,0.67713140669585,-1.55372126905196,-1.13264739051145,-1.07862273931177,2.01058093704644,2.50955640237393,0.517660006338886,-2.25886322369171,4.28786172851761,3.79999995287797,0.0,3.09999970633838,0.700000337119789,-1.60000002186294,1.99999984724822,2.09971953235275,1.38247057325857,3.60288326516122,3.43693579661333,2.65878562321184,4.13782699405849,4.65041459442163,4.68313727063017,5.79682274294369,5.37786634595851,1.18350635493289,6.44339762487436,3.50686976639459,0.0574740238200775,-4.11927606715011,-5.6937412158003,-9.87677875300243,-8.33113421099992,-2.27211699888024,0.792225060962835,-0.228301892172851,-0.0317947678412054,1.47312472702352,2.06467207892409,2.27718065334808,-9.19623149726978,8.65449786002131,5.52198580589408,2.13591131996631,2.0865743380603,2.07306882481713 +GBR,United Kingdom,,2.70131421328257,1.0986962362898,4.85954520244742,5.59481105874345,2.13033308666921,1.56744960482462,2.77573771112216,5.47269310050385,1.93913775370176,2.70858148856038,2.00453760135888,3.62637105743457,7.31352424220155,-1.67969792645223,-0.676870194049812,2.80465599545295,2.36335094409922,3.40337382855367,2.74677219709527,-2.17910220206564,-1.3234581733779,2.09306886432368,3.62434641923983,2.67092491032817,3.59938143817158,4.01232987098167,4.56208082572655,5.0321018040183,2.28144671930745,0.779266119883417,-1.29243974751705,1.29472849604493,3.49048744653493,4.95341092800692,3.53339096338078,2.61491606949487,4.8799853038024,3.24631482881618,2.94445869854798,4.52438074810904,2.39102229602824,1.70902736258567,3.21851612669614,2.32601853325993,2.7906547174688,2.18982132415704,2.93091133582351,-0.0494116727402769,-4.58740819226323,2.26054460931586,0.851437739020326,1.52826034721852,1.71846961308464,3.16017351468152,2.14040560033371,2.20652020635357,3.02322219840339,1.55133095078786,1.25629899760824,-10.047896637362,8.5431118453255,5.14970388522993,0.271650048355255,1.08027716858781,1.38844207858726 +ARG,Argentina,,5.42784287621933,-0.852021521903339,-5.30819682539126,10.1302976615431,10.5694333376394,-0.659726170629824,3.19199662339177,4.82250077932424,9.67952601078326,3.04564331917115,5.65813108184925,1.62841651825939,2.81175403849035,5.5338043957736,-0.0284121037110907,-2.01825151683394,6.93414775653342,-4.50612480201359,10.2227634638681,1.51878394990179,-5.18978914135593,-0.735659159103619,4.34909327926756,1.57073870182023,-5.18902435343904,6.15337706275693,2.70486882770558,-1.08993334133791,-7.15694965597953,-2.46721377810472,9.13311056738961,7.93729155643076,8.20697907221228,5.83620070368526,-2.84520961057079,5.52668982715234,8.1110467707457,3.85017885156228,-3.38545704063269,-0.78899893905691,-4.40883968258557,-10.8944848285903,8.83704079576924,9.02957330068152,8.85165992013437,8.04715150043027,9.00765087504757,4.05723310346406,-5.91852507634947,10.1253981561002,6.00395169280579,-1.0264204544321,2.40532378079436,-2.51261532081394,2.73115982828944,-2.08032784377811,2.81850297775918,-2.61739646282038,-2.00086100285785,-9.9004848136464,10.4418119882506,6.02074537534855,-1.85578803046339,-1.34293072157739,4.36733330217022 +USA,United States,,2.29999999989235,6.100000008077,4.40000000466181,5.8000000040378,6.40000000107759,6.50000000865072,2.49999999306594,4.80000000278329,3.09999999656958,0.216606498143278,3.29272243899894,5.25550197771585,5.64567975951033,-0.540550190362552,-0.205619191208058,5.38803390614694,4.62418650079432,5.53520648057204,3.16598822512103,-0.256775739391401,2.53770127042986,-1.80301377321203,4.58379096944093,7.2364531665454,4.16957535422245,3.46265504522061,3.45462966441372,4.17698239478443,3.67223784138015,1.88596557755378,-0.108312873733027,3.52249717254271,2.75179586314481,4.02902272637739,2.68443073970295,3.77277269047364,4.44712793442667,4.48313335499124,4.78842504836746,4.07758575997799,0.955538337599847,1.70044733078285,2.79560596378604,3.84777169267127,3.48354993710312,2.78453963674239,2.0038583039496,0.113587247501258,-2.5765002366652,2.69519258673525,1.56440685205419,2.28911339009767,2.11783009443471,2.52381981757846,2.94555045227337,1.81945147909089,2.45762230126449,2.96650506701943,2.58382533052225,-2.081375977796,6.15202247867526,2.52421385073222,2.93436322440155,2.79318715363841,2.16138195623856 diff --git a/lectures/business_cycle_data.csv.yml b/lectures/business_cycle_data.csv.yml index d36847d..2943445 100644 --- a/lectures/business_cycle_data.csv.yml +++ b/lectures/business_cycle_data.csv.yml @@ -72,7 +72,7 @@ license: # economies, which places the export after the WDI release that first carried # 2023 growth, and integrity.upstream records how far the live series has # moved since. -retrieved: null +retrieved: 2026-09-01 maintainer: QuantEcon # --------------------------------------------------------------------------- @@ -83,7 +83,7 @@ maintainer: QuantEcon # to byte-match and no figure a change here could alter today. integrity: - sha256: 20e928d444b749ae52bc6c29283c316fe0c7e9cc4a215a0303bdfb365a7bc38d + sha256: 05b52b8d66be7565a75b66f897bf21095a7844fb0d47c036cad019bfc514315f upstream: # `diverged`, not `failing`: the builder was re-run in full against live # WDI and its output differs from these bytes in the way a revised @@ -93,36 +93,10 @@ integrity: # status / date / against / note are stamped by scripts/snapshots.py on # every merged refresh (the delta block below is dropped at the same time, # since a fresh snapshot is `verified` by construction). - status: diverged + status: verified date: 2026-09-01 - against: builders/business_cycle.py # live WDI, lastupdated 2026-07-13 - note: > - Re-run of the builder against live WDI on 2026-09-01: same five rows, - same layout, same single structural null, but 236 of the 320 - overlapping cells revised and two new year columns. Expected for a - tracking snapshot; see `delta`. - delta_kind: upstream-moved - delta: > - Live WDI (lastupdated 2026-07-13) vs the committed file, measured - 2026-09-01 with wbgapi 1.0.12 / pandas 2.3.3. Shape 5x67 against 5x65: - YR2024 and YR2025 added, populated for all five economies. In the - 1960-2023 overlap, 63 of 64 year columns carry at least one revised - cell — 236 of 320 cells in all, median |change| 0.0 pp, 90th percentile - 0.29, 99th percentile 1.00, maximum 1.50 pp. Nulls unchanged (YR1960, - all five). No column renamed, no economy added or dropped, row order - unchanged. - delta_evidence: > - `upstream-moved` is the only kind possible here: the committed file is - the verbatim output of `wb.data.DataFrame(...).to_csv()` with no - post-processing, and the builder that wrote it is the one committed - beside it, so a local edit has nowhere to have happened. The pattern of - the delta — small revisions spread across the whole history, largest in - the most recent years, plus two appended years — is the signature of - national-accounts revision and rebasing (WDI aggregates on constant - 2015 prices, and member countries revise back-series), not of any - change on this side. The revisions are why validate() bounds the - overlap window at 5 pp rather than asserting equality. - register: QuantEcon/data-lectures#39 + against: builders/business_cycle.py + note: "Refreshed 2026-09-01 by the scheduled refresh: these bytes are the builder's validated output from the live source that day, so they are verified by construction. Overlap window YR1960..YR2023: 236 of 320 cells revised, max |change| 1.5002." # --------------------------------------------------------------------------- # Shape @@ -139,14 +113,14 @@ schema: columns: - {name: economy, dtype: str, description: "ISO3 code — wbgapi's index, written as the CSV's first column; the lecture reads it with index_col=0"} - {name: Country, dtype: str, description: "country name, the label column wbgapi adds with labels=True"} - - {pattern: 'YR\d{4}', dtype: float64, description: "annual real GDP growth, percent, one column per year on an unbroken grid from YR1960 to the newest year the source carries (YR2023 in the committed bytes). Observed -10.89 to 13.20"} + - {pattern: 'YR\d{4}', dtype: float64, description: "annual real GDP growth, percent, one column per year on an unbroken grid from YR1960 to the newest year the source carries (`schema.date_range.end`). The builder's validate() bounds every value at ±50 and requires at least one |value| ≥ 1, so a units switch cannot ship; the observed range is deliberately not written here, since prose in a dynamic snapshot's manifest must not embed facts a refresh can change"} row_count_floor: 5 # exact by construction: the builder asks for # five economies and validate() asserts the # set. Columns, not rows, are what grow. # `end` is the newest year column in the COMMITTED bytes, stamped by # scripts/snapshots.py on every refresh — which is what makes a refresh PR's # manifest diff self-describing. - date_range: {start: 1960, end: 2023} + date_range: {start: 1960, end: 2025} # Growth is undefined in the series' first year, so YR1960 is empty for all # five economies and nothing else is. validate() asserts exactly this hole. diff --git a/provenance/README.md b/provenance/README.md index 7c1231b..b5a0dd3 100644 --- a/provenance/README.md +++ b/provenance/README.md @@ -19,8 +19,10 @@ failing every rule a dataset must meet. | `business_cycle_metadata.md` | `builders/business_cycle.py` | `wb.series.metadata.get('NY.GDP.MKTP.KD.ZG')` — the World Bank's own record for the series: definition, source, periodicity and the `License_Type: CC BY-4.0` / `License_URL` fields that `lectures/business_cycle_data.csv.yml` cites | | `business_cycle_info.md` | `builders/business_cycle.py` | `wb.series.info(q='GDP growth')` — the fuzzy-search listing the consuming lecture teaches; kept because the builder reproduces the lecture's own query, not because anything reads it | -Files here carry no manifest and no hash gate. They are **regenerated on every -builder run** and are expected to drift: between the committed dump of +Files here carry no manifest and no hash gate. Runs of blank lines in the +upstream text are collapsed to one before writing — whitespace carries no +evidence, and the raw World Bank dump had runs ten newlines deep. They are +**regenerated on every builder run** and are expected to drift: between the committed dump of 2025-02 and a run on 2026-09-01 the World Bank rewrote the metadata prose and added a `Dataset: WB_WDI` line. The manifest's typed `source` and `license` fields are the durable record; a dump is the evidence they were read from. diff --git a/provenance/business_cycle_metadata.md b/provenance/business_cycle_metadata.md index 7f4e6b0..0b3f97f 100644 --- a/provenance/business_cycle_metadata.md +++ b/provenance/business_cycle_metadata.md @@ -3,13 +3,15 @@ Series: NY.GDP.MKTP.KD.ZG Aggregationmethod: Weighted average -------- -Developmentrelevance: An economy's growth is measured by the change in the volume of its output or in the real incomes of its residents. The 2008 United Nations System of National Accounts (2008 SNA) offers three plausible indicators for calculating growth: the volume of gross domestic product (GDP), real gross domestic income, and real gross national income. The volume of GDP is the sum of value added, measured at constant prices, by households, government, and industries operating in the economy. GDP accounts for all domestic production, regardless of whether the income accrues to domestic or foreign institutions. +Dataset: WB_WDI -------- -IndicatorName: GDP growth (annual %) +Developmentrelevance: This indicator is related to the national accounts, which are critical for understanding and managing a country's economy. They provide a framework for the analysis of economic performance. National accounts are the basis for estimating the Gross Domestic Product (GDP) and Gross National Income (GNI), which are the most widely used indicator of economic performance. They are essential for government policymakers, providing the data needed to design and assess fiscal and monetary policies; and are also used by businesses and investors to assess the economic climate and make investment decisions. NAS enable comparison between economies, which is crucial for international trade, investment decisions, and economic competitiveness. More specifically, this indicator is related to national accounts aggregates. Gross Domestic Product (GDP), Gross National Income (GNI), and other aggregates provide a snapshot of the size and health of an economy by measuring the total economic activity within a country. They can thus be used by policymakers to design and implement economic policies, as they reflect the overall economic performance and can indicate the need for intervention in certain areas. Aggregates also allow for comparisons between different economies, which can be useful for trade negotiations, investment decisions, and economic benchmarking. By examining aggregates over time, economists and analysts can identify trends, cycles, and potential areas of concern within an economy, and investors can use national accounts aggregates to assess the potential risks and returns of investing in a particular country. Overall, national accounts aggregates are fundamental tools for economic analysis, policy formulation, and decision-making at both the national and international levels. +-------- +IndicatorName: GDP (annual % growth) -------- License_Type: CC BY-4.0 -------- -License_URL: https://datacatalog.worldbank.org/public-licenses#cc-by +License_URL: https://datacatalog.worldbank.org/int/public-licenses#cc-by -------- Limitationsandexceptions: Each industry's contribution to growth in the economy's output is measured by growth in the industry's value added. In principle, value added in constant prices can be estimated by measuring the quantity of goods and services produced in a period, valuing them at an agreed set of base year prices, and subtracting the cost of intermediate inputs, also in constant prices. This double-deflation method requires detailed information on the structure of prices of inputs and outputs. @@ -23,14 +25,19 @@ Rebasing of national accounts can alter the measured growth rate of an economy a To obtain comparable series of constant price data for computing aggregates, the World Bank rescales GDP and value added by industrial origin to a common reference year. Because rescaling changes the implicit weights used in forming regional and income group aggregates, aggregate growth rates are not comparable with those from earlier editions with different base years. Rescaling may result in a discrepancy between the rescaled GDP and the sum of the rescaled components. To avoid distortions in the growth rates, the discrepancy is left unallocated. As a result, the weighted average of the growth rates of the components generally does not equal the GDP growth rate. -------- -Longdefinition: Annual percentage growth rate of GDP at market prices based on constant local currency. Aggregates are based on constant 2015 prices, expressed in U.S. dollars. GDP is the sum of gross value added by all resident producers in the economy plus any product taxes and minus any subsidies not included in the value of the products. It is calculated without making deductions for depreciation of fabricated assets or for depletion and degradation of natural resources. +Longdefinition: Gross domestic product is the total income earned through the production of goods and services in an economic territory during an accounting period. It can be measured in three different ways: using either the expenditure approach, the income approach, or the production approach. This indicator denotes the percentage change over each previous year of the constant price (base year 2015) series in United States dollars. -------- Periodicity: Annual -------- -Source: World Bank national accounts data, and OECD National Accounts data files. +Referenceperiod: 1961-2025 -------- -Statisticalconceptandmethodology: Gross domestic product (GDP) represents the sum of value added by all its producers. Value added is the value of the gross output of producers less the value of intermediate goods and services consumed in production, before accounting for consumption of fixed capital in production. The United Nations System of National Accounts calls for value added to be valued at either basic prices (excluding net taxes on products) or producer prices (including net taxes on products paid by producers but excluding sales or value added taxes). Both valuations exclude transport charges that are invoiced separately by producers. Total GDP is measured at purchaser prices. Value added by industry is normally measured at basic prices. When value added is measured at producer prices. - -Growth rates of GDP and its components are calculated using the least squares method and constant price data in the local currency. Constant price in U.S. dollar series are used to calculate regional and income group growth rates. Local currency series are converted to constant U.S. dollars using an exchange rate in the common reference year. +Source: Country official statistics, National Statistical Organizations and/or Central Banks; +National Accounts data files, Organisation for Economic Co-operation and Development (OECD); +Staff estimates, World Bank (WB) +-------- +Statisticalconceptandmethodology: Methodology: National accounts are compiled in accordance with international standards: System of National Accounts, 2008 or 1993 versions. Specific information on how countries compile their national accounts can be found on the IMF website: https://dsbb.imf.org/ +Statistical concept(s): The conceptual elements of the SNA (System of National Accounts) measure what takes place in the economy, between which agents, and for what purpose. At the heart of the SNA is the production of goods and services. These may be used for consumption in the period to which the accounts relate or may be accumulated for use in a later period. In simple terms, the amount of value added generated by production represents GDP. The income corresponding to GDP is distributed to the various agents or groups of agents as income and it is the process of distributing and redistributing income that allows one agent to consume the goods and services produced by another agent or to acquire goods and services for later consumption. The way in which the SNA captures this pattern of economic flows is to identify the activities concerned by recognizing the institutional units in the economy and by specifying the structure of accounts capturing the transactions relevant to one stage or another of the process by which goods and services are produced and ultimately consumed. -------- Topic: Economic Policy & Debt: National accounts: Growth rates +-------- +Unitofmeasure: %