feat(item108): Ibis join_asof strategy semantics (forward/nearest support) - #305
Merged
Merged
Conversation
…ward)+emulation, retire IB-REL-13 Probe evidence (polars 1.43.2 | ibis 12.0.0): ALL PASS (scripts/probes/probe_asof_emulation.py)
… + colliding-name crash fix
…5 entry, regenerate all generated artifacts Added assert_predicate_capability_gated() to tests/fixtures/capability_gating.py: capability_gate()/assert_capability_gated() only resolve static family/dialect/param facts, never predicate facts (which need actual bound call values to evaluate) - the new helper is the predicate-fact counterpart, keeping the raw pytest.raises form out of migrated test files per test_no_migrated_site_carries_a_raw_capability_form. hatch run mypy:check confirmed pre-existing broken on develop (identical 98 errors, same duplicate-module-name root cause) - zero new errors from this change (diffed).
test_json_completeness caught a real gap: _fact_dict (render_markdown.py) never serialized CapabilityFact.predicate at all, so test_json_completeness's own JSON-vs-live-model fact-multiset comparison hardcoded () for every fact's predicate term (previously harmless - no production predicate fact existed to expose it). Item 108's IB-REL-15 predicate fact is the first one, and tripped the drift the hardcoded () was silently hiding. Added a kind-tagged JSON encoding (_operand_json/_clause_dict) mirroring schema._operand_key's own tagging, plus the test-side inverse (_operand_key_from_json/_clause_key_from_json) so the identity round-trips exactly through the wire format. Full tests/relations + tests/core sweep (8141 passed) confirms no other predicate-fact-shaped gaps remain.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements spec
2026-08-18-ibis-asof-strategy-design.md(draft 2) / plan2026-08-18-ibis-asof-strategy.md(draft 5, narrowed-scope architecture after 4 rounds of adversarial GLM-5.3 review).Closes the silent wrong-data path on Ibis (
Relation.join_asof(strategy="forward"|"nearest")previously silently returnedbackward's results on every Ibis dialect). Fixes the narwhals-pandasnearestgenuine-cross-side-tie defect and a colliding-payload-name crash. Fixes a schema leak ({on}_right) and a nondeterminism bug (duplicate-left-row order varying run-to-run) in the native Ibis backward path. RetiresIB-REL-13(ibis-sqlitegainsjoin_asofsupport for every strategy via an Ibis-relational-expression emulation).Three narrow, non-value-affecting divergences (duplicate-tie winner and interleaved-
by-group row order on Ibis SQL dialects and narwhals-pandas) are declared viaDivergenceFact/xfail_divergence— the codebase's existing mechanism for dozens of comparable backend quirks — rather than chased with increasingly intricate emulation code. Every declared divergence is backed by a probe-verifiedDivergenceFactand anxfail_divergence-marked test that would catch a silent behavior change.Architecture
backwarduses nativeasof_joinonibis-duckdb/ibis-polars(closest to the original design).forward/nearest(any SQL dialect) and every strategy onibis-sqliteroute through an Ibis-relational-expression emulation (inner candidate join → windowed rank → left-keep join).strategyjoinedgate_params; a new predicateCapabilityFactpermanently gatesforward/nearestonibis-polars(non-equality join predicates are rejected there).pandasnearestuses a dual backward/forward join + distance comparison to fix the genuine cross-side tie (Polars' documented forward-wins rule).New divergences (declared, not fixed)
IB-REL-15—ibis-polarspermanently rejectsforward/nearest(non-equality predicates unsupported).IB-REL-16—ibis-duckdbnativebackwardpicks the FIRST duplicate-right-key row on a tie; Polars picks the LAST.IB-REL-17— Ibis SQL-backend paths group output bybyvalue; Polars preserves left input order for interleaved groups.IB-REL-14—ibis-sqlitehas noTimestampDeltatranslation; temporalnearest/toleranceunsupported there.NW-REL-03/NW-REL-04/NW-REL-05— narwhalstolerancegate, null-key raise, andnearestduplicate-tie-winner divergence on the pandas-family dialects.IB-REL-13retired (closed inregistry/upstream-issues.yaml, not deleted, per theIB-REL-11precedent).Verification
scripts/probes/probe_asof_emulation.py):polars 1.43.2 | ibis 12.0.0— ALL PASS, including a 10-rep determinism check on the native duckdb path.tests/relations/cross_backend/test_rel_join_results.py,test_rel_extension_ops_results.py,test_rel_operation_keys.py, capability registry/predicate-probe/census/integrity/enforcement, divergence facts, upstream registry join/validate/audit — all green.tests/relations/+tests/core/— 8141 passed, 0 failed, 68 skipped (pre-existing), 720 xfailed (expected divergences).ruff checkclean.mypydiffed byte-identical against pristinedevelop(98 pre-existingimport-untyped/duplicate-module errors, zero new — confirmed unrelated to this change).Review history
GLM-5.3 design review (job 016): Sound with required changes. Plan reviews (jobs 017–019): 3 rounds, each finding new bugs concentrated in one place — code chasing exact Polars parity on things Polars itself treats as an implementation convention (duplicate-tie winner, interleaved-group row order), never in the actual value-matching logic (correct in every round). User directed a scope narrowing: fix the real defects, declare the narrow edge cases. Draft 5 reflects that decision and was independently re-verified against the installed libraries (not just review prose) before implementation began.