Skip to content

feat(labeling): produce native keyed multi-horizon outcomes - #114

Merged
stefan-jansen merged 13 commits into
mainfrom
feat/native-forward-outcomes
Oct 21, 2026
Merged

stefan-jansen merged 13 commits into
mainfrom
feat/native-forward-outcomes

Conversation

@stefan-jansen

@stefan-jansen stefan-jansen commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Native outcome preparation

Closes #110. PR #113 is merged as 9dcd4b0c97f0776d8ac8a40dc33c6ac767e93ed8.
The branch now includes its actual main ancestry and is ready for final hosted qualification.
Both workflows reuse one precise arithmetic kernel; this is not a replacement precision PR.

Add ml4t.engineer.labeling.forward_returns, returning a normal keyed long Polars table.
Keep existing fixed-label signatures/defaults and the sole Alphalens facade in Diagnostic.
Engineer does not clean/bucket factors, fill quotes, acquire data or reconstruct adjustments.

The producer supports multiple named horizons, sparse exact entry keys, observation/expected-bar/
session/elapsed units, per-asset/common clocks, cumulative/exit-bar returns, exact nanosecond
targets and tolerance, explicit tail/missing-price policies, known-at availability and price basis.
Original row/quote masks from externally prepared research views survive entry/reference/exit
selection. Inserted fill rows cannot clear named-calendar completeness checks.

Reuse the existing schedule/grid adapters and future-price lookup. Extend owned LabelSpec fields
with shared FeedSpec serialization, preserving calendar omission/null and old wire keys when unused.
No new shared contract, graph runtime, adjustment service or Alphalens namespace.

Reproductions and implementation

  • Original main c69e7badaac012ad98b1c4b8aa9a67a952b08f5c lacks native multi-horizon production.
    Existing public workflows reject a declared 1ns grid and lose outcome declarations at persistence.
  • Public elapsed-window cases reproduced microsecond rounding (102 instead of the correct101.5).
    Shared future-price lookup reproduced ns/us join mismatch; it now retains exact durations/keys.
  • Existing UInt64 fixed labels reproduced returns around1.8e17 for a decline that should be-.1.
    Shared integer-safe arithmetic repairs this without changing established floating definitions.
  • An actual installed outcome wheel reproduced a positive horizon wrapping into the past at the
    nanosecond limit. Out-of-range inputs/targets now fail atomically.
  • Decimal price production reproduced zero tiny returns. Preserve scale through subtraction,
    then convert to Float64. Decimal return output dtype changes explicitly; original price columns
    and existing Float32/Float64 definitions stay unchanged. Opposite-sign limits are covered.
  • Real early-close, holiday, DST and overnight cases distinguish expected bars, session deadlines
    and physical targets. A covered session with no full eligible bar remains a missing endpoint,
    not a skipped session or an outside-tail assertion.

Qualification so far

Current implementation head: 3c916aeeabd3742f4ae833d019a42e069e82c0ae.
It has exactly the previously qualified 8732c89 tree
7705f10d88fc222ea81c8a80a57ab5e3ffdc4f4a after resolving the two prerequisite-history
conflicts. No source behavior changed during that integration.

  • Focused refreshed arithmetic/outcome/artifact/time-horizon cases: 456 passed.
  • Fresh full configured suite at committed 3c916ae: 5,094 passed, 57 configured performance deselections, 17 warnings in 207.06 seconds.
    The actual critical coverage gate passes with 96.05% line and 90.31% branch coverage.
  • Ruff lint/format, ty, actual commit hooks, executable guide workflows and strict MkDocs pass.
  • Exact 3c916ae wheel SHA256 930a46661b34ea9b00306eefbf7e8b94a1f72680150f6b30b071b0d70d8a2d88;
    sdist SHA256 018d5eb2d43d7e5c4668f10aa9bd85bd67b301e82b47e9ed8ae88041fc85c2d6.
  • Actual isolated producer -> Parquet/LabelSpec JSON -> Diagnostic prepared panel passes ordinary
    cumulative and reference-key exit-bar values against the receiver's public price route. Adjacent
    nanosecond keys/exits, string and compound stable IDs, availability and persisted timedelta replay
    survive the native handoff. All 201 Engineer and 289 Diagnostic installed package files were
    byte verified; five receiving implementations independently match source 643b94b. Both wheels
    are unpublished qualification artifacts, not a claim about published Diagnostic 0.1.9.
  • The issue verification gate covers all nine obligations with zero waivers or deferrals.
    Producing-code mutations detect contaminated asset keys, wrong clocks, filled-grid acceptance,
    quote filling, lost availability, ignored calendars, wrong endpoint method, rounded durations,
    numeric scale loss, calendar omission loss and non-replayable horizon serialization.
  • The new producer is included in the existing critical coverage gate. No thresholds, failure
    checks, supported-platform matrix or security gate were waived.

Owner review of the actual 15-file final diff found no remaining source finding. Before merge,
every applicable final-head hosted source/artifact/platform/security/docs gate must pass. Fresh
full source and rebuilt installed-artifact convergence pass at 3c916ae; earlier unchanged-tree
evidence remains scoped to its stated build.
No release, tag or deployment is requested.

AI assistance: Codex implemented code, tests and documentation, reproduced the numerical defects,
and ran/reviewed the checks above. No independent human review or publication is claimed.

Select native arithmetic at schema planning time, subtract widened integer ticks before floating division, and retain float dtype/formula and logarithmic domains. Require Polars 2.0 for the native schema dispatch API. Add eager/lazy/grouped regression cases and an executable migration recipe.

Refs #107
Retain endpoint column roles, units, method and basis declarations plus canonical FeedSpec serialization in optional label records. Preserve legacy wire keys when new fields are absent. Add JSON/YAML/null-omission and nanosecond Parquet round-trip controls.

Refs #110
The required dependency audit on PR113 reproduced GHSA-6q48-6c7w-9gw2 in certifi 2026.7.22. Upgrade only certifi to patched 2026.10.19; both runtime and complete contributor lock exports now pass pip-audit. Existing dependency PR106 retains the affected version and does not supply this repair.
Use exact epoch ticks for declared grid and session membership, retaining authoritative calendar boundaries. Preserve compound entity keys without hashing them. Public regressions demonstrate missing bars, submicrosecond endpoints and elapsed-window rounding before repair.

Refs #110
Preserve requested and actual endpoints, physical nanosecond durations, calendar/session units, common-clock versus per-asset indexing, availability and explicit missingness. Share precise integer arithmetic with fixed labels to prevent unsigned wrap. Extend owned artifact declarations without changing legacy wire keys and include an executable native preparation recipe.

Refs #110
The fresh contributor audit reproduces eight Pillow12.3 advisories published after the earlier clean local audit. Lock the patched13.0 release without ignoring vulnerabilities or changing runtime requirements. Core requirements are unchanged; focused numerical and executable documentation checks pass.
An installed-wheel probe reproduced positive nanosecond horizons wrapping into the past. Reject unrepresentable inputs and targets atomically, parse canonical fixed durations without pandas unit aliases, and cover invalid inputs plus calendar/unit conflicts through public calls. Include the new outcome producer in the existing critical coverage gate.

Refs #110
Accept explicit paired original-row and observed-quote masks without filling any price. Carry entry, reference and exit flags; validate named grids on actual original rows so synthetic fill rows cannot conceal a missing bar. Preserve the source mask declarations in existing LabelDefinition and reject contradictory inputs atomically.

Refs #110
Public probes reproduce scale-truncated Decimal outcomes and a covered early-close session misreported as tail data. Preserve exact decimal subtraction before floating returns, including opposite-sign limits; keep empty scheduled sessions in the requested count. Add direct overnight, DST, calendar precedence and row-spacing regressions and document the Decimal output dtype boundary.

Refs #110
Actual Timedelta outcome values could persist in display syntax rejected by the public API. Emit canonical exact nanosecond values and reuse one fixed-duration conversion for targeting, tolerance and persistence. The public output-to-LabelSpec-to-JSON-to-replay regression fails before repair and passes afterward; document qualified consumer versions and exact-key lineage.

Closes #110
PR113 is merged as9dcd4b0. Retain the qualified outcome arithmetic and Decimal documentation while adding its actual main ancestry. The resulting tree exactly matches reviewed8732c89; no source behavior changes. Refs #110.
@stefan-jansen
stefan-jansen marked this pull request as ready for review October 20, 2026 23:03
Copilot AI balanced review requested due to automatic review settings October 20, 2026 23:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@stefan-jansen
stefan-jansen merged commit 35b0160 into main Oct 21, 2026
49 checks passed
@stefan-jansen
stefan-jansen deleted the feat/native-forward-outcomes branch October 21, 2026 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Produce precise multi-horizon outcomes and Diagnostic handoff

2 participants