Skip to content

feat(typespec): parameterized backend_type fidelity + raise on invalid (item 54, gap 1) - #294

Merged
discreteds merged 6 commits into
developfrom
feature/typespec-param-backend-type-fidelity
Aug 17, 2026
Merged

feat(typespec): parameterized backend_type fidelity + raise on invalid (item 54, gap 1)#294
discreteds merged 6 commits into
developfrom
feature/typespec-param-backend-type-fidelity

Conversation

@discreteds

Copy link
Copy Markdown
Member

Summary

Item 54, PR-1 (gap 1 + validation strictness): the canonical resolver _resolve_field_native silently dropped parameterized backend_type strings on Polars/Narwhals/PyArrow (Ibis/Pandas were already correct), and silently absorbed invalid backend_type strings into a canonical fallback.

This PR:

  • Adds core/dtypes/_paramstring.py — a restricted AST-validated parser for constructor-call-style dtype reprs (Polars/Narwhals str(dtype) output), shared by both targets. Closed whitelist namespace; adversarial-tested (no eval of unvalidated input).
  • Upgrades parse_type_string on Polars and Narwhals (safe-eval parser) and PyArrow (bounded bracket/paren regex family: timestamp/duration/time/decimal) — parameterized dtypes like Datetime(time_unit='us', time_zone='UTC'), Decimal(precision=38, scale=10), List(Int64), Array(Int64, shape=(5,)) now round-trip to real native dtypes instead of canonical fallback.
  • Adds InvalidBackendTypeError and wires it into _resolve_field_native: a non-empty, non-None backend_type the target cannot parse now raises (names field + string + target) instead of silently degrading. None/"" fall through unchanged (item 53's ANY→STRING path preserved).
  • Flips TestInlineReadItem54DeferredTestInlineReadParameterizedBackendType (the deliberate flip item 53 left for item 54).

Spec: 2026-08-16-converters-resolver-dtype-fidelity-design.md (rev 2, GLM-5.2 adversarial review folded). Plan tasks 1–6, RED-first TDD, one commit per task.

Verification

  • Full suite: 22935 passed, 0 failed
  • hatch run ruff:check: clean
  • hatch run mypy:check: 0 new errors (98 pre-existing, all import-untyped/fixture-class errors present at baseline; touched files contribute only pre-existing pyarrow import-untyped)
  • Grep-confirmed: no production FieldSpec(backend_type=...) string literal exists; only test usages (all parseable or intentional raise cases)

@discreteds
discreteds merged commit 2816f6d into develop Aug 17, 2026
4 checks passed
@discreteds
discreteds deleted the feature/typespec-param-backend-type-fidelity branch August 17, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant