Strong independent STEO provenance milestone (20 Oct 2026): Catalyst Cooperative Zenodo source archive v2.0.0, published May 26 2025, contains original STEO XLSX from 2022–2024. Verified real independent full 36-source comparison #39444963696: 2022 12/12, 2023 12/12, 2024 12/12 = 36/36 original XLSX byte-identical SHA-256 between the May2025 third-party deposit and EIA files acquired Oct2026, with three annual ZIP MD5 checks against Zenodo's deposited manifest. Full original source file SHA and provenance report. PR #51 merged (290/290 offline + 3/3 disposable PG PASS). No claim of original source-day 2022–24 PIT: an archive published May2025 cannot rule out earlier revisions before deposit, and the 316 historical forecast comparisons remain descriptive, not investable or predictive edge. No Neon writes, scheduler, real trades or paper reallocations, MyDevil/Mac source backup closed, M1/M2/M3 HOLD_NO_CHANGE.
STEO data provenance clarified (20 Oct 2026): PR #49 merged (283/283 offline tests + 3/3 isolated PG integration PASS). Official EIA correction notices show actual republication of some 2023–25 archived issues, affecting identified tables 3, 4 and 10, while other notices cover text, PDF-only, supplements or a future fix. 9-source curated issue-specific official notice evidence report and read-only
scripts/steo_official_notice_audit.pydistinguish notice/repost, affected scope, and actual WTI/Brent Table2 verification: no confirmed original-day historical XLSX bytes (PIT), and no proven Table2 change or absence of change from these notices. All 316 historical EIA forecast/reference pair results retain RECONSTRUCTED research-only status; no Neon writes, schedules or trades, M1/M2/M3 HOLD_NO_CHANGE. Operational source backup stays CLOSED.
Untouched earlier STEO source-era test COMPLETE (20 Oct 2026): real EIA official 32-file run #39433488201 SUCCESS: 24 archived 2022–23 issues + 8 Jan–Aug24 reference-only issues, 144 source WTI/Brent price-forecast pairs on frozen +1/+3/+6-month EIA vs same-issue previous-month naive benchmarks. Older EIA has lower mean absolute price error in all six 24-pair pooled groups, BUT 2022 EIA wins 6/6 while 2023 EIA wins only 2/6 short-horizon groups; naive wins the other four, matching the later 2024–26 long-horizon pattern. Strong regime dependence, not a stable forecast/investment edge. Older 2024 bridge source original XLSX SHA matched 8/8 prior October-2026 archival reads, historical as-published source bytes remain NOT PIT-certified. All authentic 32 SHA receipts and per-year results. PR #48 merged 2018+ source Table2 year headers, 276/276 offline + 3/3 isolated PG tests PASS. No Neon changes, schedules, secrets, real orders or virtual allocation changes; M1/M2/M3 HOLD_NO_CHANGE.
Historical STEO robustness VERIFIED (20 October 2026): PR #44 merged (269 offline + 3 isolated PG integration tests PASS), real official 34/34 archived EIA XLSX walk-forward source run #39428981358 SUCCESS, 34/34 exact RAW SHA repeat-match, 172 source forecast/reference pairs but only 62 distinct instrument–target months, and 104 matured-label as-of eligible comparisons. The naive/EIA dynamic method-switcher did not beat the better fixed method in any of six asset-horizon groups (worse in two). Later reference revisions: Brent 0 observed, WTI up to $0.10/bbl on 1 pair in each of two horizon groups; overlap of months prevents counting unique revisions from group aggregates. All 6 actual dynamic/fixed MAE groups, sources and methodological limits. Reconstructed vintage bytes first published historically NOT independently PIT certified; descriptive research only, M1/M2/M3 HOLD_NO_CHANGE, no Neon changes, orders or schedules.
Full actual official historical EIA STEO study COMPLETED (20 October 2026): one-shot 34/34 publicly sourced XLSX run #39426001269 SUCCESS with 172 paired WTI/Brent 1/3/6-month forecast-vs-later STEO reference comparisons. Source EIA retrospective MAE lower than same-issue naive only at 1 month (WTI 6.44 vs 7.46 USD/bbl, Brent 7.76 vs 8.58); naive lower at 3 months (WTI 10.18 vs EIA 10.87, Brent 10.89 vs 12.21) and 6 months (WTI 12.21 vs 15.53, Brent 12.56 vs 16.99). Crucially overturns the misleading 12-pair pilot's all-horizons EIA lead. Full evidence with each source XLSX SHA and vintage/horizon/year counts. Retrospectively reconstructed only: historical exact source bytes PIT unverified, reference spot price ≠ tradable PLN total-return or investment performance; no new Neon writes, orders, schedules or virtual allocations; M1/M2/M3 HOLD_NO_CHANGE.
Real historical STEO forecast evaluation (20 Oct 2026): PR #42 merged (262 offline + 3 isolated PostgreSQL tests PASS); actual public official EIA run #39425055719 produced 12 paired Jan24/Jan25 WTI/Brent monthly STEO forecasts versus later-edition historical prices; 1/3/6-month USD/bbl MAE documented here. n=2 for each commodity-horizon, so NO predictive edge or portfolio implication established. PR #43 merged (262 offline + 3 disposable PG PASS): one-time full 34-edition Jan24–Oct26 archival research job #39426001269 in progress; no scores to report until completion. All sources use today-hosted archived XLSX, historical original-publication bytes NOT PIT-certified, no Neon writes or real orders; M1/M2/M3 HOLD_NO_CHANGE.
Historical analysis can start NOW (20 Oct 2026): genuine Neon STAGING market-history data contain 3,746 legacy reference price/FX rows across 12 series; descriptive correlation and anomaly diagnostics already ran READ ONLY, independent first-publication PIT remains unverified. EIA officially lists 34 distinct historical STEO PDF/XLSX monthly releases from Jan 2024 to Oct 2026. PR #39 merged after 251/251 offline tests + 3/3 disposable PostgreSQL tests:
scripts/steo_historical_archive.pyexposes an official release-date catalog, manual bounded EIA XLSX capture to operator-selected private local archive, byte SHA-256 receipts, independent offline replay and sheet-name inventory. Real archived XLSX extraction/forecast-accuracy scores have NOT been executed yet. Next operator pilotjan24,jan25,oct26and evidence-driven parser design; no Neon writes, no change to M1/M2/M3 paper HOLD_NO_CHANGE. Manual operator guide.
Research milestone (20 October 2026): EIA operational backup is CLOSED / USER-CONFIRMED after a 6-file Mac mini copy matched 3/3 original HTML SHA-256 and 3/3 manifests; MyDevil sources passed independent offline verification. No paid OVH bucket required for current simulation. Backup acceptance and precise limits. PR #38 merged (242/242 offline + 3/3 disposable PG integration PASS): read-only
scripts/eia_prospective_archive_research.pyindependently reparses archived official HTML+manifest vintages and compares overlapping date/price revisions only between truly acquired snapshots. First captures = baseline only, not historical PIT. Operator offline runbook. Virtual M1/M2/M3 remain HOLD_NO_CHANGE; zero Neon writes or orders.
Actual 3-source EIA private archive now captured (20 October 2026): operator on MyDevil FreeBSD confirmed WTI, Brent and Henry Hub raw original HTML files + timestamped manifests written to a private local POSIX directory with
immediate_independent_replay=truefor all three; WTI passed an additional separate offlineverify. Actual SHA-256, acquired-at timestamps and limits. These NEW 20 October captures are real; earlier 19 October CI source requests still had no retained original HTML. Private local archive is NOT WORM. No Neon source/version writes, roles, credentials, schedule or trades. Historical PIT remains unverified; virtual M1/M2/M3 HOLD_NO_CHANGE.
MyDevil archive readiness (20 October 2026): PR #37 merged after 235/235 offline + 3/3 disposable PostgreSQL integration tests. New
scripts/eia_archive_storage_preflight.pyinspects an already existing private POSIX archive root READ ONLY without logging paths or creating files. This does not verify actual MyDevil quota, ZFS restore, persistence or WORM; SSH unavailable in current runtime, so no MyDevil remote check was performed. OVHcloud Object Storage Object Lock is a candidate independent WORM backend, not configured or approved. Live Neon versioned EIA source/price rows 0, writer role absent. Read-only/MyDevil + OVH plan.
Source-byte provenance upgrade — PR #36 merged (19 October 2026): Official EIA original HTML can now be retained in an operator-selected private local POSIX archive, keyed by raw SHA-256, with a distinct exact-time acquisition receipt and independent offline replay/semantic SHA. 228/228 offline + 3/3 disposable PostgreSQL tests passed. Local no-overwrite is NOT certified WORM; no historical live EIA bytes have been archived, no Neon source/version writes, no new roles. Operator capture/verify and limits. Virtual models remain HOLD_NO_CHANGE.
Actual live EIA receipts completed (2026-10-19): WTI ↔ STAGING 695/695 exact matches, 0 missing and 0 price mismatches, historical first-660 SHA unchanged; all three official EIA snapshots validated via public HTTPS (WTI 285, Brent 289, Henry Hub 283), original HTML and semantic SHA recorded. No Neon write: EIA versioned docs/observations both 0, EIA writer role absent. Current-source validation ≠ original historical PIT publication proof;
MARKET_DATA_NOT_READY, paper M1/M2/M3HOLD_NO_CHANGE. Complete dated evidence.
EIA source integrity hardening (19 October 2026): PR #33 merged: original HTML SHA and separate canonical date/price SHA, 14-day source freshness rejection, strict NO UPDATE/DELETE/TRUNCATE writer ACL. 210 offline + 3 disposable PostgreSQL tests passed, main CI green. Actual Neon STAGING READ-ONLY check still 695 WTI, 706 Brent, 3 pre-existing documents and 3 pre-existing version rows; dedicated EIA writer role absent. No new live EIA↔Neon manual 695/695 source receipt, no live versioned ingestion, and no verified historical first-publication vintages. Research-only HOLD_NO_CHANGE. Handoff.
Versioned EIA source ingestion core — main, 19 October 2026. PR #32 merged: 204/204 synthetic offline and 2/2 isolated PostgreSQL 16 integration tests PASS. The official-source CLI is public HTTPS dry-run only; the source/version SQL writer requires a new dedicated least-privilege STAGING role which does not yet exist in Neon. No source/version rows were written: WTI 695 legacy, Brent 706, versioned 3 source docs + 3 synthetic price fixtures unchanged. No verified original historical PIT vintages,
MARKET_DATA_NOT_READY, M1/M2/M3HOLD_NO_CHANGE. Operator/security review.
Current 695-day WTI audit on main (19 October 2026): PR #31 merged after 195/195 offline tests. The manual-only post-import WTI EIA ↔ Neon STAGING receipt checks 695 exact source/stored date+price matches, immutable original 660-row fingerprint and real 2026-10-19 backfill timestamps. The live post-import job has not yet been dispatched; PR CI is not evidence of actual source agreement. No verified historical individual publication vintages or PIT import,
MARKET_DATA_NOT_READY, M1/M2/M3HOLD_NO_CHANGE.
19 October 2026 — STAGING legacy WTI coverage FIXED (not PIT). With explicit user authorization, 35 missing EIA WTI current-archive reference rows were inserted into the isolated STAGING legacy
investment_lab.price_observationstable. Independent SQL postflight: 695/695 WTI days, all 35 new prices match the official EIA table, 0 mismatches, original 660-row digest unchanged; Brent 706 unchanged,price_observation_versionsandsource_documentsremain only the three old synthetic fixtures. Historical publication vintages still unverified;MARKET_DATA_NOT_READYand M1/M2/M3HOLD_NO_CHANGE. The earlier frozen receipt (expected 660/35 gap) is now ARCHIVAL; do not rerun as a current-state test. Full handoff / Issue #22.
Educational, research-only Opportunity & Decision Intelligence. Reproducible market-data validation, historical anomaly research, falsifiable hypotheses, evidence-for/against and hypothetical portfolio comparisons. No actual trades, brokerage/exchange execution or investment recommendations.
Read AGENTS.md, then the complete session handoff. It contains current verified project state, architecture, Neon staging vs production, GitHub CI runs, key decisions, remaining blockers and a ready-to-reuse continuation prompt.
Supporting documentation:
- Opportunity Radar and retrospective research
- Legacy-history read-only bridge and role authorization
- Issue #4 — full implementation history and backlog
- Draft PR #1 — original full beta package, not merged
The repository already contains Opportunity Radar, non-overlapping research evaluation, an evidence decision ledger, macro/geopolitical context validation, an aggregate-only historical research bridge, Neon staging preflight, data audit and manual educational research canaries. Real Neon staging read-only/PIT and full research cycle smoke tests have succeeded, but market-data readiness remains blocked.
Updated 19 October 2026: After explicit user approval, the separate least-privilege historical reader isl_legacy_history_reader was created on the isolated Neon staging branch as NOLOGIN, using the reviewed SQL. Independent checks confirm SELECT only on investment_lab.price_observations, no writes, unchanged operational-agent ACL, and no corresponding production role. LOGIN/password/GitHub secret/manual historical workflow have NOT been enabled and require separate explicit authorization. This documentation change does not authorize those actions.
Original virtual models M1/M2/M3 depend on verified PLN total-return equity/bond reference history; legacy PRICE_ONLY spot/reference observations cannot substitute for those indices. Retrospective research remains labelled UNVERIFIED_HISTORICAL_PUBLICATION.
After a second explicit user approval, isl_legacy_history_reader has PostgreSQL LOGIN enabled on staging, but no password (rolpassword IS NOT NULL=false) and no GitHub environment secret. The Neon connector password-reset attempt failed and was independently checked; the role still has historical-reference SELECT-only rights. The manual workflow remains blocked. See safe operator credential bootstrap — never paste secrets into this repository or a chat.
After user's approval and rediscovery of the successful 18 October bootstrap from Issue #3, server-side SQL generated an undisclosed random initial password for isl_legacy_history_reader. Direct Neon PostgreSQL checks now confirm LOGIN=true, password present=true, SELECT only on the historical-reference table, no writes/private holdings and production unchanged. Still pending: reset this unknown bootstrap password to a private final password in Neon Console, then configure the separate GitHub Environment secret NEON_HISTORICAL_READ_DATABASE_URL. No historical workflow, schedules or trades have run. Follow the operator bootstrap/reset guide. Previous status paragraphs are retained as chronological snapshots.
The manual staging-only GitHub historical-reference run #39241685091 completed successfully with the dedicated historical reader credentials. The aggregated research report artifact was inspected: 3,711 reference observations, 8 analyzable series, 62 retrospective anomaly candidates, 47 nonoverlapping research windows, four investigation-only series (Brent, WTI, Henry Hub, ETH). No validated macro events or predictive performance; historical publication vintages unverified. M1/M2/M3 remain HOLD_NO_CHANGE; zero orders, database writes or allocation changes and no recurring schedule. No new credentials or resets needed. Data provenance/benchmarks are the next blockers, not the already-working staging secret. Prior status paragraphs are chronological and may describe earlier blockers.
- Python: deterministic historical validation, anomaly detection, evidence grading and hypothesis checks; LLM may assist with explanations, not invent source evidence.
- Neon PostgreSQL: versioned point-in-time sources, append-only research journal, separate legacy historical reference data.
- GitHub Actions: offline regression tests and manual-only staging checks; no unattended investment-related schedule.
- Artifacts: research-only JSON/Markdown aggregate diagnostics, fingerprints, caveats and benchmark evidence. Never publish raw licensed prices or private positions.
This repository is public. Never commit passwords, DSNs containing credentials, tokens, seed phrases, wallets, account data, private portfolio exports or licensed raw datasets. Production Neon is outside the permitted test scope. Never widen the journal agent role or apply the staged historical-reader role SQL without fresh, explicit approval. Preserve verified TLS, least-privilege ACL and careful publication chronology.
python -m unittest discover -s tests -v
python -m scripts.opportunity_demo_fixture --output /tmp/isl-synthetic.json
python -m scripts.opportunity_research_pipeline --input /tmp/isl-synthetic.json --out-dir /tmp/isl-educational-reportThe sample data above are entirely artificial. Passing the offline tests is not proof of forecasting skill or production readiness. Consult the full session handoff before continuing.