Read the six folded datasets from data-lectures, not high_dim_data - #292
Merged
Merged
Conversation
Seven reads across four files. The six datasets moved into
QuantEcon/data-lectures (dl#62), so every read here has to follow them --
and the HOST has to change too, not just the org and repo.
media.githubusercontent.com is the LFS *media* endpoint and routes per
path by LFS status. These files were LFS-tracked in high_dim_data and are
plain git in data-lectures, so the media host 404s for all six. Measured:
the media URL for data-lectures returns 404 with 0 bytes, the raw URL
returns 200. A mechanical org/repo swap that kept the host would have
broken every chart in these three lectures.
All seven now read raw.githubusercontent.com, including the two that used
the github.com/*/raw/ redirect form and would have survived a bare org
swap. One spelling across all four consuming repos is deliberate: the
set 1/2 repoints broke lecture-wasm by "harmonising" its URLs onto the
redirect form, whose 302 carries an empty access-control-allow-origin and
so is rejected by the browser before it is followed (dl#46). If every
repo spells it the same way, that class of fix cannot break anything.
_static/lecture_specific/inequality/data.ipynb is edited by hand and is
the read nothing mechanical would ever carry: no audit scans it, the
build never executes it, and the translation sync is .md-only, so this
repo's copy could never receive a sync PR for it. It is also served --
it returns 200 on this repo's Pages site with the URL in its body.
No prose is touched. The "2020" claims about the Forbes billionaires data
are contradicted by the bytes and need fixing, but that is content: it
will arrive through the sync PR that the English repoint generates, which
is what the sync is for.
Verified:
- both acceptance greps clean: no media-host data-lectures read, and no
high_dim_data reference anywhere in lectures/
- all 7 URLs return 200
- three-way byte identity on the four cross_section files: manifest
sha256 = new URL = old high_dim_data URL. The SCF pair was verified
against high_dim_data's LFS objects in dl#62
- the 7 `# i18n` markers from #291 all survive
- notebook still parses as JSON; every code cell still compiles, apart
from three pre-existing IPython magic cells (one !pip, two %%time)
that are byte-identical on main
No CI in any repo can see these reads -- this repo is outside the audit's
SCAN_REPOS by decision. Checked by hand.
Step C1 of QuantEcon/workspace-lectures#23. Data: QuantEcon/data-lectures#62
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for playful-platypus-17d3bb ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Pull request overview
This PR updates hard-coded dataset download URLs in several Chinese lecture sources to reflect the migration of six “folded” datasets from QuantEcon/high_dim_data (some via the LFS media host) into QuantEcon/data-lectures as plain git files, using the raw.githubusercontent.com host to avoid redirect/CORS and LFS-media 404 issues.
Changes:
- Repoint SCF dataset reads in
mle.mdandinequality.mdfromhigh_dim_data/ redirect-form URLs toraw.githubusercontent.com/QuantEcon/data-lectures/.... - Repoint three
cross_sectiondataset reads inheavy_tails.mdfrom the LFS media host toraw.githubusercontent.com/QuantEcon/data-lectures/.... - Update the served notebook artifact
lectures/_static/lecture_specific/inequality/data.ipynbto the new SCF dataset URL.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| lectures/mle.md | Updates SCF mini dataset URL to data-lectures via raw.githubusercontent.com. |
| lectures/inequality.md | Updates SCF mini dataset URL to data-lectures via raw.githubusercontent.com. |
| lectures/heavy_tails.md | Updates Forbes + cities dataset URLs from LFS media host to raw.githubusercontent.com in data-lectures. |
| lectures/_static/lecture_specific/inequality/data.ipynb | Updates the embedded SCF URL in the notebook so the served artifact doesn’t point at a dead link. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seven reads across four files. The six datasets moved into
QuantEcon/data-lectures(QuantEcon/data-lectures#62), so every read here follows them — and the host changes too, not just the org and repo.This is a hand-written PR because nothing mechanical can produce it: this repo sits outside the data audit's
SCAN_REPOSby decision, so no CI in any repo can see these reads. Checked by hand.Why the host has to change
media.githubusercontent.comis the LFS media endpoint and routes per path by LFS status. These six were LFS-tracked inhigh_dim_dataand are plain git indata-lectures, so the media host 404s for every one of them. Measured today:media.githubusercontent.com/media/QuantEcon/data-lectures/main/lectures/SCF_plus_mini.csvraw.githubusercontent.com/QuantEcon/data-lectures/main/lectures/SCF_plus_mini.csvSo a mechanical org/repo swap that preserved the host would have broken every chart in these three lectures — silently, since nothing here executes them on a PR.
Why all seven land on
raw.githubusercontent.comFive were on the media host and had no choice. The other two —
mle.mdandinequality.md— used thegithub.com/…/raw/redirect form, which routes per path by LFS status and would have survived a bare org swap. They are normalised anyway, so that all four consuming repos spell the same read the same way.That is deliberate. The set 1/2 repoints broke
lecture-wasmby "harmonising" its URLs onto the redirect form (QuantEcon/data-lectures#46) — reasonable-looking, since the two spellings did look like an inconsistency, but that form's 302 carries an emptyaccess-control-allow-origin, so a browser rejects it before following the redirect. Nothing caught it: wasm's build does no execution, the pages still returned 200, and the figures simply never appeared. If every repo spells it one way, that class of fix cannot break anything again.The notebook is the read nothing would ever carry
_static/lecture_specific/inequality/data.ipynb:37is edited by hand. No audit scans_static/**, the build never executes it, and the translation sync is.md-only — so this repo's copy could never receive a sync PR for it, no matter how many times the English source is repointed. It is also a served artifact: it returns 200 on this repo's Pages site with the URL in its body, so leaving it stale would keep serving a dead link to anyone who downloads it.No prose is touched
The lectures claim the Forbes billionaires data is a 2020 snapshot. The bytes contradict that — 282 distinct timestamps spanning 2020-04-07 to 2023-04-15, and the chart plots real-time values rather than the annual list. (The Global 2000 "2020" claim nearby is correct; its builder hardcodes the 2020 endpoint.)
That is content, not a repoint, so it is deliberately left out of this PR. It will arrive through the sync PR that the English repoint generates — which is what the sync is for, and it gets the translation right rather than having me hand-patch Chinese prose.
Verification
data-lecturesread, and nohigh_dim_datareference anywhere inlectures/cross_sectionfiles — manifestsha256= new URL = oldhigh_dim_dataURL, so this changes the address and not the data. The SCF pair was verified againsthigh_dim_data's LFS objects in Fold in the six high_dim_data datasets (PR B1) data-lectures#62# i18nmarkers from Mark the seven localised code lines in heavy_tails and mle with# i18n#291 all survive — this PR edits the same cells they protect!pip, two%%time) that are byte-identical onmainStep C1 of QuantEcon/workspace-lectures#23. C2 (intro + wasm + the canary, 21 reads) follows.
🤖 Generated with Claude Code