Skip to content

String match strategies (PREFIX, SUFFIX, CONTAINS) fail on pandas/narwhals backends #89

Description

@discreteds

Summary

The starts_with, ends_with, and contains expression methods fail on pandas-based backends when used with per-row column references. The pandas str accessor receives an Expr object where it expects a plain string.

Discovered during cross-backend performance benchmarking of mountainash-utils-rules.

Affected backends and strategies

Strategy polars pandas narwhals-polars narwhals-pandas ibis-duckdb ibis-sqlite
PREFIX
SUFFIX
CONTAINS

Error messages

pandas / narwhals-pandas:

TypeError: expected a string or tuple, not Expr
  .venv/.../pandas/core/strings/accessor.py:3172: TypeError

narwhals-polars:

TypeError: cannot create expression literal for value of type Expr.
Hint: Pass `allow_object=True` to accept any value and create a literal of type Object.

Root cause

The starts_with, ends_with, and contains compile methods produce column-reference expressions for per-row pattern matching (e.g., matching a context value against each rule's string pattern). The polars and ibis backends handle column references natively in string operations, but:

  • pandas passes the Expr object directly to pd.Series.str.startswith() / .endswith() / .contains(), which expect scalar strings
  • narwhals wraps pandas and inherits the same limitation; narwhals-polars fails on PREFIX/SUFFIX but not CONTAINS (suggesting partial support)

Reproduction

From mountainash-utils-rules (branch release/26.4.0):

hatch run test:test-perf-target "tests/test_benchmarks.py::TestStrategyIsolation" -v -k "pandas and PREFIX" --benchmark-only

Workaround

mountainash-utils-rules benchmarks skip these combinations:

_STRING_MATCH_STRATEGIES = {MatchStrategy.PREFIX, MatchStrategy.SUFFIX, MatchStrategy.CONTAINS}
_STRING_MATCH_BROKEN_BACKENDS = {"pandas", "narwhals-pandas", "narwhals-polars"}

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions