feat(capabilities): wire extract/extract_boolean, strptime_timestamp.timezone, retire MA duplicates - #290
Merged
Conversation
…timezone, retire MA duplicates
Backlog item 62. Two canonical Substrait datetime ops plus one options fix,
landed as a single commit (the closed-by-default option-disposition matrix
couples the MA-retirement drain to the same commit as the wiring — see notes).
extract / extract_boolean (new):
- component (23/2-member closed domains, DatetimeComponent/BooleanComponent),
indexing (ONE/ZERO omission-sentinel, calendar components only), timezone
(open IANA_TIMEZONE value class) wired through protocol -> API builder ->
three backends -> capability facts -> disposition matrix.
- Backend bodies rewritten per semantic probe (gate disabled, all 4 fixtures):
deleted every silent fallback (return x.dt.year() / pl.lit(False));
in-domain-but-unsupported components now raise BackendCapabilityError,
backed by UNSUPPORTED facts (polars 6, ibis 7, narwhals 9 declared
components; IS_DST declared on all three).
- Two additional silent-wrong bugs found and fixed: MICROSECOND/NANOSECOND
were returning since-last-SECOND (0-999999/0-999999999) instead of Substrait's
since-last-unit (0-999) semantics — now % 1000; narwhals QUARTER formula
(month // 4 + 1) was wrong for months 7/10/11 — now (month-1)//3+1.
TIMEZONE_OFFSET declared everywhere (native primitives require tz-aware
input extract can't guarantee at build time — input-type-dependent,
matches spec §7.3, not a raw backend exception).
strptime_timestamp.timezone (end-to-end fix):
- MountainAshScalarStringAPIBuilder.to_datetime gains timezone (IANA-
validated), threading through the def's options tuple (now
("format","timezone")) to the protocol and all three backends.
- narwhals body wired for real (dt.replace_time_zone); ibis silently drops
(no timezone primitives, matching assume_timezone/to_timezone/
local_timestamp/extract.timezone) — now backed by an UNSUPPORTED fact
instead of a silent discard.
Mountainash-namespace retirement:
- Deleted MA extract/extract_boolean (protocol + 3 backend impls) —
dead code, positional-only, no caller (named wrappers call backends
directly). Drains their option-matrix parks by deletion.
Capability facts (new capabilities/datetime/extract.py + additions to
value_classes_substrait.py):
- Closed option_value= grain (component/IS_DST) lives in extract.py.
- Open value_class=IANA_TIMEZONE grain (extract/extract_boolean/
strptime_timestamp timezone) lives in value_classes_substrait.py per the
placement-guard's domain-module restriction (spec §3: domain modules never
carry value_class facts).
Disposition matrix (test_arg_types_datetime.py + option_disposition.py):
full per-backend x per-value dispositions, probes (declared cells verified
via gate-disabled native probe + strict xfail), invalid-sentinel rejections,
and TESTED_OPTION_PARAMS entries for all 6 wired option params. Drained all
6 "operation not implemented" parks plus the 2 MA argument-channel aliases.
Conformance/reachability drift entries retired (signatures/options/call-
patterns that were "known divergent" now genuinely match): protocol
alignment KNOWN_ASPIRATIONAL, signature/options drift dicts, unverifiable-
call-pattern entries, and A2 local builders + reachability builders added
for the two newly-public FKEYs.
AST-identity coverage added: extract/extract_boolean node shape
(test_ast_scalar_datetime.py), to_datetime timezone option presence/
omission (test_ast_str_to_date.py).
Regenerated docs/reference/expression-coverage.{md,json} (3 new
declarations from extract.py).
Deleted tests/expressions/backends/test_narwhals_extract_capability_error.py
— exercised the now-retired MA extract's inline ISO_WEEK raise; the same
guarantee (ISO_WEEK on narwhals raises BackendCapabilityError, not
AttributeError) is now covered by the extract disposition matrix's declared
cell + strict-xfail probe.
Deferred (spec §7, unchanged from plan): strptime_time + precision (needs
core/dtypes work); is_dst (item 65, IS_DST stays UNSUPPORTED); input-type-
dependent overload validity (error normalization, pre-existing).
Verification: full suite 22519 passed / 0 failed / 81 skipped / 3734 xfailed;
ruff clean; mypy — no new errors (2 touched-file hits are the pre-existing
ibis import-untyped baseline noise, same class as the other 95). Smoke-
tested end-to-end on all 4 fixtures (extract, extract_boolean, indexing,
declared-component raise, strptime_timestamp+timezone) via the public
relation API.
Deviation from the written plan: the plan sliced this into PR-1 (extract/
extract_boolean) and PR-2 (strptime_timestamp.timezone + MA retirement).
The option-disposition matrix's closed-by-default reason-staleness check
(test_untested_option_param_custom_reasons_still_hold) ties MA retirement
to the same commit as wiring extract into the API builder — draining the
MA park's "not implemented in API builder" reason requires the MA methods
to already be gone the moment extract becomes a builder method name, or
the check fails on the newly-false reason. Landed as one commit; still two
logically-separable diffs within it (extract+extract_boolean vs
strptime_timestamp+retirement) per the plan's task boundaries.
Backlog: substrait-datetime-missing-ops (item 62)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backlog item 62 (substrait-datetime-missing-ops). Wires the two canonical Substrait datetime ops
extract/extract_booleanplus an end-to-end fix forstrptime_timestamp.timezone, retires the dead-code Mountainash-namespaceextract/extract_booleanduplicates, and closes two additional silent-wrong bugs discovered by the semantic probe.Spec:
mountainash-central/04.planning/mountainash/superpowers/specs/2026-08-15-substrait-datetime-missing-ops-design.md(rev 2, codex-reviewed)Plan:
mountainash-central/04.planning/mountainash/superpowers/plans/2026-08-15-substrait-datetime-missing-ops.md(rev 2, GLM-5.2-reviewed)Landed as one commit — see the commit message for why the plan's PR-1/PR-2 split collapsed (the closed-by-default option-matrix reason-staleness check ties the MA retirement to the same commit as wiring
extractinto the API builder).What changed
extract/extract_boolean: new, fully wired (protocol → builder → 3 backends → capability facts → disposition matrix).component(23/2-value closed domains),indexing(ONE/ZERO),timezone(open IANA value class).x.dt.year()/pl.lit(False)); declared components now raiseBackendCapabilityError, backed by facts. Also fixed MICROSECOND/NANOSECOND (were since-second, should be since-unit per Substrait) and narwhals' wrong QUARTER formula.strptime_timestamp.timezone: now reaches the backend end-to-end (str.to_datetime(fmt, timezone=...)); narwhals wired for real, ibis declared (no timezone primitives).extract/extract_booleanretired: dead code, no caller, drains their parks by deletion.test_signature_conformance.py/test_protocol_alignment.py/test_api_reachability.pyare now genuinely aligned and removed.extract/extract_booleanandto_datetimetimezone option presence/omission.docs/reference/expression-coverage.{md,json}.tests/expressions/backends/test_narwhals_extract_capability_error.py— tested the retired MA method; same guarantee now covered by the disposition matrix.Verification
ruff check: clean.mypy: no new errors (2 touched-file hits are the pre-existing ibis import-untyped baseline noise).Deferred (per spec §7, out of scope here)
strptime_time+precision(needscore/dtypeswork) — future item.is_dstreal implementation — item 65.