Skip to content

fix(demo): vendor 2008 hindcast data excluded by .gitignore - #5

Merged
hongjin-he merged 4 commits into
hongjin-he:mainfrom
Ajay03299:impl/vendor-2008-demo-data
Aug 3, 2026
Merged

fix(demo): vendor 2008 hindcast data excluded by .gitignore#5
hongjin-he merged 4 commits into
hongjin-he:mainfrom
Ajay03299:impl/vendor-2008-demo-data

Conversation

@Ajay03299

Copy link
Copy Markdown
Contributor

Fixes #1.

demo/hindcast_2008.py — the walk-forward replay featured in the README — failed with FileNotFoundError on any fresh clone. The unanchored *.csv rule in .gitignore matches at every depth, so the four fixtures the script expects in demo/data/2008/ were silently skipped by git add and never committed, despite the module docstring stating they are meant to be vendored.

Changes

.gitignore — adds !demo/data/**/*.csv after the *.csv rule so the demo fixtures are tracked. Scoped narrowly; no other CSV path is affected.

demo/data/2008/*.csv — the four fixtures, ~225 KB total, all public and keyless:

File Source
ff_factors_0309.csv Ken French Data Library — daily research factors
ind10_0309.csv Ken French Data Library — 10 industry portfolios, value-weighted
ted.csv FRED TEDRATE
vix.csv FRED VIXCLS

demo/data/fetch_2008.py — regenerates the fixtures from source. Committing data without the code that produces it makes provenance unverifiable, so the script ships alongside. It handles the two format hazards in the French files: the 10-industry file contains a second equal-weighted section (truncated at its marker, otherwise every date would appear twice and corrupt rolling(63).corr()), and missing data is encoded as -99.99, which would survive /100.0 as a −99.99% return — the script raises instead. Row parsing whitelists on ^\d{8}, rather than a fixed skiprows, so a change to the upstream preamble fails loudly instead of shifting the header into the data.

Verified deterministic: re-running the fetcher into a clean directory produces byte-identical output to the committed files.

tests/test_hindcast_data.py — 15 tests, no network. The most important asserts the fixtures are git-tracked, not merely present on disk; that is the precise condition that failed here, and a test for file existence alone would have passed on the author's machine. Also covers date-index alignment between the two French files (they feed vol21 and corr63 separately, so a mismatch would reindex to NaN and silently degrade Λ rather than raise), required columns, sentinel values, and window coverage. An end-to-end test runs the demo and asserts it exits 0 while reproducing its headline result; it restores figures/hindcast_2008.png afterward so the run is non-destructive.

.github/workflows/ci.yml — adds pandas matplotlib to the explicit install. They were only reaching CI through pip install -r requirements.txt || true, which silently no-ops if any pin fails to resolve — and requirements.txt pins jax[cuda12] and apache-airflow, neither of which is reliable on a GPU-less runner. Test dependencies shouldn't depend on a best-effort install.

Verification

$ python demo/hindcast_2008.py
eval sample: 2004-04-01 → 2009-12-31  (1449 trading days)
θ = 2.0 (composite 2σ) · sustain rule = 5 consecutive closes
first close  > θ : 2007-08-10
first SUSTAINED  : 2007-08-16   → 272 trading days before Lehman
false positives before 2007: 0 days above θ
✓ figures/hindcast_2008.png

Matches the README's stated result. pytest tests/ -q → 65 passed (50 existing + 15 new) on Python 3.13, macOS arm64.

Notes

  • I went with vendoring over a fetch-on-first-run script since it preserves the demo's zero-setup, keyless property, and the docstring indicates that was the intent. Happy to switch to fetch-only if you'd prefer to keep data out of the repo — the script is already written either way.
  • Redistribution: the Ken French files are freely published for research use and attributed in both the fetch script and the demo docstring. Say the word if you'd rather not vendor them.
  • Out of scope, noticed while working: requirements.txt pins jax[cuda12], which can't install on non-NVIDIA machines (it blocked my local setup on Apple Silicon), and setup.sh assumes conda plus Docker containers for TimescaleDB and Kafka. Neither is needed to run the demos. Worth a separate issue if useful.

@Ajay03299
Ajay03299 force-pushed the impl/vendor-2008-demo-data branch from 7ae22bc to c5972d0 Compare July 30, 2026 21:51
The unanchored `*.csv` rule in .gitignore silently excluded the four
public-data fixtures that demo/hindcast_2008.py expects in
demo/data/2008/, so the demo failed with FileNotFoundError on any
fresh clone.

- Add `!demo/data/**/*.csv` negation so vendored fixtures are tracked
- Vendor the four CSVs (~225 KB total, all public and keyless)
- Add demo/data/fetch_2008.py, the script that regenerates them from
  Ken French's Data Library and FRED

Verified: `python demo/hindcast_2008.py` exits 0 and reproduces the
README's headline result — sustained signal 2007-08-16, 272 trading
days before Lehman, 0 false positives before 2007.

Fixes hongjin-he#1
Guards hongjin-he#1 by asserting the fixtures are git-tracked (not merely present
on disk), that the two French files share an identical date index, that
no missing-data sentinels survive parsing, and that hindcast_2008.py
exits 0 while reproducing its headline result.
Test dependencies were only reaching CI via `pip install -r
requirements.txt || true`, which silently no-ops if any pin fails to
resolve (requirements.txt pins jax[cuda12] and apache-airflow). The new
hindcast fixture tests import pandas and invoke a matplotlib-based demo,
so both are now installed on the explicit, failing line.
Generalizes the guard past the 2008 fixtures. The same .gitignore rule
also dropped the seven CSVs documented in demo/data/memory2026/SOURCES.md,
so the check walks demo/data/ rather than naming individual files.
@Ajay03299
Ajay03299 force-pushed the impl/vendor-2008-demo-data branch from c5972d0 to d11d3bd Compare August 2, 2026 17:43
@hongjin-he
hongjin-he merged commit 6435f11 into hongjin-he:main Aug 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

demo/hindcast_2008.py fails on fresh clone - vendored 2008 CSVs excluded by .gitignore

2 participants