Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 37 additions & 9 deletions docs/known-divergences.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

<!-- DO NOT EDIT — this file is auto-generated by scripts/generate_divergences_catalog.py -->

> Last generated: 2026-08-13 03:40 UTC
> Last generated: 2026-08-19 05:24 UTC

## Overview

Total divergences tracked: **125**
Total divergences tracked: **128**

## Summary

Expand Down Expand Up @@ -62,6 +62,7 @@ Total divergences tracked: **125**
| IB-DT-07 | ibis-sqlite: microsecond() returns seconds instead of microsecond component | ibis-sqlite | upstream_bug | Strict xfail | Investigating |
| IB-DT-08 | ibis backends: nanosecond() returns 0 — no sub-microsecond precision from Python datetime | ibis-duckdb, ibis-polars, ibis-sqlite | by_design | Strict xfail | By Design |
| IB-DT-09 | Ibis today() upcasts date to timestamp and now() uses query-time UTC SQL | ibis-duckdb, ibis-polars, ibis-sqlite | by_design | Accepted (documented) | By Design |
| IB-DT-19 | SQLite backend memtable registration crashes on null date/timestamp values (NaT binding) | ibis-sqlite | upstream_bug | None | Needs Filing |
| NW-DT-01 | Datetime offset ops require literal integers — column expressions rejected (parameter-width) | narwhals-pandas, narwhals-polars | parameter_width | Enhanced error message | Open |
| NW-DT-02 | week_of_year not supported on pandas backend | narwhals-pandas | upstream_feature_gap | Strict xfail | Investigating |
| NW-DT-03 | dt.time() is not supported on any Narwhals dialect — NotImplementedError | narwhals-pandas, narwhals-polars, narwhals-lazy | upstream_feature_gap | Strict xfail | Investigating |
Expand Down Expand Up @@ -122,7 +123,8 @@ Total divergences tracked: **125**
| IB-REL-10 | ibis-sqlite: drop_nans/unpivot/melt raise OperationNotDefinedError | ibis-sqlite | upstream_feature_gap | Strict xfail | Needs Filing |
| IB-REL-11 | RESOLVED: ibis-duckdb asof join returned nondeterministic row order | ibis-duckdb | mountainash_internal | None | Resolved (internal) |
| IB-REL-12 | cross_join between two same-named tables from different Ibis connections raises a column-binding error on SQL backends | ibis-duckdb, ibis-sqlite | mountainash_internal | Strict xfail | Investigating |
| IB-REL-13 | ibis-sqlite: asof join raises UnsupportedOperationError | ibis-sqlite | upstream_feature_gap | Strict xfail | Needs Filing |
| IB-REL-13 | RESOLVED: ibis-sqlite asof join raised UnsupportedOperationError | ibis-sqlite | upstream_feature_gap | None | Resolved (internal) |
| IB-REL-15 | ibis-polars rejects non-equality join predicates, blocking the join_asof forward/nearest emulation | ibis-polars | upstream_feature_gap | Enhanced error message | Needs Filing |

### Aggregate Operations

Expand Down Expand Up @@ -765,6 +767,19 @@ Total divergences tracked: **125**
| Workaround | Accepted (documented) |
| Last Verified | 2026-07-05 |

### IB-DT-19: SQLite backend memtable registration crashes on null date/timestamp values (NaT binding)

| Field | Value |
|---|---|
| Project | ibis |
| Category | Datetime Operations |
| Root Cause | upstream_bug |
| Affected Backends | ibis-sqlite |
| Status | Needs Filing |
| Workaround | None |
| Notes | _register_in_memory_table (ibis/backends/sqlite/__init__.py) stages memtables via op.data.to_frame() (a pandas roundtrip). A null date32 or timestamp value becomes pandas NaT, which sqlite3 cannot bind — crashes create_table/insert for any frame containing a null temporal value. Verified empirically 2026-08-18 against ibis 12.0.0 — raw pyarrow.Table sources are incidentally safe for date columns only (PyArrow's Table.to_pandas() defaults date_as_object=True, so a null date32 becomes Python None, not NaT) but not for timestamp columns; polars/pandas DataFrame sources crash for both date and timestamp columns (their own to_pandas() maps Date to datetime64 directly). Discovered downstream in mountainash-data (DEBT-13), not via mountainash's own cross-backend probe suite — mountainash itself does not yet have a CapabilityFact/DivergenceFact exercising this path. |
| Last Verified | 2026-08-18 |

### NW-DT-01: Datetime offset ops require literal integers — column expressions rejected (parameter-width)

| Field | Value |
Expand Down Expand Up @@ -1264,7 +1279,7 @@ Total divergences tracked: **125**
| Notes | ibis-sqlite lacks array/pivot relational translations: Relation.drop_nans()/ unpivot()/melt() raise OperationNotDefinedError; ibis-duckdb/ibis-polars and polars/narwhals compute them. Renamed from a colliding DivergenceFact id (was "IB-REL-07", which already named an unrelated entry above — the ArrayValue.sort() descending-parameter gap) when backlog item 87's id-collision detector caught the reuse; the DivergenceFact and this entry now self-reference via upstream_ref (backlog item 87, 2026-08-13). |
| Last Verified | 2026-08-13 |

### IB-REL-11: RESOLVED ibis-duckdb asof join returned nondeterministic row order
### IB-REL-11: RESOLVED: ibis-duckdb asof join returned nondeterministic row order

| Field | Value |
|---|---|
Expand All @@ -1274,7 +1289,7 @@ Total divergences tracked: **125**
| Affected Backends | ibis-duckdb |
| Status | Resolved (internal) |
| Workaround | None |
| Notes | Originally described as "asof join is unreliable on Ibis SQL backends" covering both ibis-duckdb and ibis-sqlite. Root-caused 2026-08-18 (PR #297 CI investigation, repeated single-test process runs flipped xfail/XPASS ~50/50): Relation.join_asof() on the Ibis backend never applied an explicit ORDER BY, and DuckDB gives no row-order guarantee for ASOF JOIN output absent one — the query planner nondeterministically returned rows in original or shuffled order run-to-run. This was not a value divergence: join values were always correct, only the row order backing the test's positional assertions flapped. Fixed by ordering the Ibis join_asof result by the join key(s) (relsys_ib_ext_ma_util.py) to match polars/narwhals' left-order preservation. The distinct ibis-sqlite raise (no ASOF JOIN translation) was split off to IB-REL-13, which remains open. |
| Notes | Originally described as "asof join is unreliable on Ibis SQL backends" covering both ibis-duckdb and ibis-sqlite. Root-caused (2026-08-18, PR #297 CI investigation): Relation.join_asof() on the Ibis backend never applied an explicit ORDER BY, and DuckDB gives no row-order guarantee for ASOF JOIN output absent one — the query planner nondeterministically returned rows in original or shuffled order run-to-run, flipping the strict-xfail test between XFAIL and XPASS(strict) independent of any test-suite change. Not a value divergence — the join values were always correct, only positional-assertion row order flapped. Fixed by ordering the Ibis join_asof result by the join key(s) (relsys_ib_ext_ma_util.py) to match polars/narwhals' left-order preservation. The distinct ibis-sqlite raise (SQLite has no ASOF JOIN translation) was split off to IB-REL-13, which remains a genuine open gap. Renamed from a colliding DivergenceFact id (was "IB-REL-08") 2026-08-13; that history stands, this entry is now closed. |
| Last Verified | 2026-08-18 |

### IB-REL-12: cross_join between two same-named tables from different Ibis connections raises a column-binding error on SQL backends
Expand All @@ -1290,18 +1305,31 @@ Total divergences tracked: **125**
| Notes | Verified empirically (backlog item 87, 2026-08-13): Relation.cross_join() raises duckdb BinderException / sqlite OperationalError ("no such column") when the two input tables share a table name but originate from separate Ibis connections (exactly the shape produced by the test suite's BackendDataFrameFactory, which names every table "test_table"); ibis-polars and polars/narwhals are unaffected. Root cause not yet isolated to Ibis vs. mountainash's join visitor — likely a missing table-alias disambiguation when two DatabaseTable operations share a name across connections. Renamed from a colliding DivergenceFact id (was "IB-REL-09", whose yaml entry described a different, no-longer-reproducible "suffixes kwarg" TypeError) when backlog item 87's id-collision detector caught the reuse; the DivergenceFact and this entry now self-reference via upstream_ref. |
| Last Verified | 2026-08-13 |

### IB-REL-13: ibis-sqlite: asof join raises UnsupportedOperationError
### IB-REL-13: RESOLVED: ibis-sqlite asof join raised UnsupportedOperationError

| Field | Value |
|---|---|
| Project | ibis |
| Category | Relational Operations |
| Root Cause | upstream_feature_gap |
| Affected Backends | ibis-sqlite |
| Status | Resolved (internal) |
| Workaround | None |
| Notes | Split from IB-REL-11 (2026-08-18) after the ibis-duckdb portion (a nondeterministic row-order bug, not a value divergence) was fixed in mountainash by ordering the join_asof result deterministically. This entry tracked the remaining genuine upstream gap: Ibis has no ASOF JOIN translation for the SQLite backend. Resolved (2026-08-19, item 108): every strategy on ibis-sqlite now routes through an Ibis-relational-expression emulation (inner candidate join + windowed rank + left-keep join) instead of native asof_join, matching Polars on every value (scripts/probes/ probe_asof_emulation.py). Two narrower row-order/tie-winner divergences from the emulation and the ibis-duckdb native path are tracked separately as IB-REL-16/IB-REL-17. The DivergenceFact was removed from divergences.py in the same change (Task 3); this entry stays as the historical record, per the IB-REL-11 precedent above. |
| Last Verified | 2026-08-19 |

### IB-REL-15: ibis-polars rejects non-equality join predicates, blocking the join_asof forward/nearest emulation

| Field | Value |
|---|---|
| Project | ibis |
| Category | Relational Operations |
| Root Cause | upstream_feature_gap |
| Affected Backends | ibis-polars |
| Status | Needs Filing |
| Workaround | Strict xfail |
| Notes | Split from IB-REL-11 (2026-08-18) after the ibis-duckdb portion (a nondeterministic row-order bug, not a value divergence) was fixed in mountainash by ordering the join_asof result deterministically. This entry tracks the remaining genuine upstream gap: Ibis has no ASOF JOIN translation for the SQLite backend, so Table.asof_join() raises UnsupportedOperationError on ibis-sqlite. Use a polars or narwhals backend for asof joins on that data. |
| Last Verified | 2026-08-18 |
| Workaround | Enhanced error message |
| Notes | item 108: Relation.join_asof(strategy="forward"|"nearest") lowers to an Ibis-relational-expression emulation (inner candidate join on a non-equality directional predicate, e.g. right.on >= left.on). The Ibis Polars backend raises TypeError("Only equality join predicates supported with pandas") for any non-equality join, so the emulation cannot run there (probe- confirmed on ibis 12.0.0/polars 1.43.2). A CapabilityFact (relations/backends/capabilities/ibis.py) gates this permanently via a predicate on the `strategy` param, raising a clear BackendCapabilityError before the raw TypeError would surface. `backward` is unaffected (native asof_join, no candidate join needed). |
| Last Verified | 2026-08-19 |

## Aggregate Operations

Expand Down
8 changes: 7 additions & 1 deletion docs/reference/expression-coverage-scoped.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

Scoped deviations — dialect, parameter, option, value-class; function-level coverage and matrices live in [`expression-coverage.md`](expression-coverage.md).

Declarations: 43 · Facts: 1466 · Registered operations: 326 · Implementation records: 978
Declarations: 43 · Facts: 1467 · Registered operations: 326 · Implementation records: 978

Legend — scoped deviations:

Expand Down Expand Up @@ -1984,6 +1984,12 @@ Legend — scoped deviations:
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| * | tolerance | — | — | unsupported | gate | build | tolerance is not None | join_asof(tolerance=...) is not supported by the Narwhals backend | Drop tolerance= or use the Polars backend. | — | 2026-07-05 | — | — |

### `JOIN_ASOF` × ibis (RKEY_MOUNTAINASH_REL)

| Dialect | Param | Option values | Value class | Level | Enforcement | Boundary | Condition | Message | Workaround | Upstream | Since | Native errors | Probe-exempt |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ibis-polars | strategy | — | — | unsupported | gate | build | — | join_asof forward/nearest lowers to a non-equality candidate join; the ibis Polars backend rejects non-equality join predicates (TypeError: Only equality join predicates supported with pandas). | Use ibis-duckdb/ibis-sqlite, or polars/narwhals backends. | IB-REL-15 | 2026-08-18 | — | — |

### `READ_RESOURCE` × polars (RKEY_MOUNTAINASH_REL)

| Dialect | Param | Option values | Value class | Level | Enforcement | Boundary | Condition | Message | Workaround | Upstream | Since | Native errors | Probe-exempt |
Expand Down
Loading
Loading