Repository navigation
feat(features): port finite Pipeline factors and causal selection workflows - #115
Merged
Merged
Conversation
Select native arithmetic at schema planning time, subtract widened integer ticks before floating division, and retain float dtype/formula and logarithmic domains. Require Polars 2.0 for the native schema dispatch API. Add eager/lazy/grouped regression cases and an executable migration recipe. Refs #107
The required dependency audit on PR113 reproduced GHSA-6q48-6c7w-9gw2 in certifi 2026.7.22. Upgrade only certifi to patched 2026.10.19; both runtime and complete contributor lock exports now pass pip-audit. Existing dependency PR106 retains the affected version and does not supply this repair.
The fresh contributor audit reproduces eight Pillow12.3 advisories published after the earlier clean local audit. Lock the patched13.0 release without ignoring vulnerabilities or changing runtime requirements. Core requirements are unchanged; focused numerical and executable documentation checks pass.
Preserve native TA-Lib defaults while providing separately registered finite weighting, Cutler RSI, population volatility and technical field definitions. Keep missing-value denominators, warmup and grouped lazy execution explicit. Refs #108.
Keep missing pairs on their original clock, expose actual regression fields through the optional stats extra, and separate civil weekdays from publication instants. Reproduce and repair integer-tick and overflow errors in finite variance and range methods without changing existing native defaults. Refs #108.
Two positive weights imply unbiased variance (x1-x0)^2/2 regardless of their ratio. Reproduce cancellation and false NaN results before accumulating the equivalent cross-weight denominator directly. Correct the earlier constant-price test assumption to zero variance. Refs #108.
…ectness differences
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port finite Pipeline feature calculations and dated selection workflows to native Polars without a legacy graph runtime. Add separately named finite methods instead of changing Wilder RSI, recursive EMA, portfolio drawdown or other existing definitions.
Closes #108.
Closes #109.
Correctness and migration evidence
Actual installed Zipline Reloaded 3.2.0 comparisons include 101 finite/return-factor/default/factory profiles, 19 selection profiles, four original/custom momentum PipelineEngine journeys through the production shifted pricing loader, 55 math/summary/return/event/operator profiles and 28 classifier/filter profiles. Each original/custom journey compares 135 exact output keys, RSI and long/short selections, including known missing quotes and listing/delisting lifetimes. Actual installed ML4T Data history output preserves a causal mean under a future correction and stable ID through split/cash/rename history.
The regression tests also cover large integer/Decimal prices, variance scale, exact nanosecond knowledge boundaries, real missing scheduled bars versus explicit row mode, keyed target alignment, input masks versus final screens, undefined multi-output values, typed Parquet and JSON/YAML handoffs, and explicit pandas presentation. Deliberate producing-code controls pass all seventeen factor/panel acceptance items with no waivers. Independent mathematical expectations supplement legacy comparisons.
One legacy correctness difference is deliberate: MaximumFilter's float-bit integer ordering selects the wrong result for all-negative groups. Native top(1) retains correct numeric ordering and stable ties. Weekday event counts remain distinct from exchange schedules. RateOfChangePercentage is signed percentage points; PercentChange uses a fractional absolute denominator. The migration reference documents these differences.
Verification status
At final source
bc2042430ab134bde7fe725878028ad7de3bf224, the full native suite passes 5,555 tests, with 57 configured deselections. Ruff lint/format, ty, executable documentation, strict MkDocs and commit hooks pass. The preceding4e20f54has an identical runtime source subtree and passes no-JIT coverage with 95.91% release-critical line coverage and 90.27% branch coverage. The final source additionally repairs actual Windows cp1252 example output and optional-SciPy executable documentation. Isolated installed Python 3.15.0rc1 with SciPy absent passes 458 focused core/compatibility/documentation cases, with 14 explicit optional-dependency skips, not a module-wide skip. Both current security audits pass.The downloaded hosted candidate
9264c27e6d09c2db2f98dd119bd9571c60e4efd5has exactly the reviewed source tree. Its wheel SHA256 isa96f8e4a4a015f0960094091d931817562e49344d24390dcdf2e41ad3eacdf4a. All four real PipelineEngine journeys and 55 expression/operator profiles also pass from that installed hosted artifact. Hosted supported-platform checks remain authoritative; merge waits for every applicable final-revision check, including the final Build Package job. Earlier failed and passing receipts retain their exact tested revisions.This PR does not publish a release, run arbitrary legacy Terms, select source revisions or adjustments, implement Alphalens cleaning/analytics, or claim order/account/performance equivalence. Optional market/loading preparation and final cross-library migration acceptance remain tracked separately by #111 and #112.
AI assistance
Codex assisted with implementation, tests, migration documentation and this PR text. Producing source, regression controls, actual installed workflows and current-head checks are reviewed and verified in this owner session; automated text or import-only checks are not treated as review completion.