Summary
The starts_with, ends_with, and contains expression methods fail on pandas-based backends when used with per-row column references. The pandas str accessor receives an Expr object where it expects a plain string.
Discovered during cross-backend performance benchmarking of mountainash-utils-rules.
Affected backends and strategies
| Strategy |
polars |
pandas |
narwhals-polars |
narwhals-pandas |
ibis-duckdb |
ibis-sqlite |
| PREFIX |
✅ |
❌ |
❌ |
❌ |
✅ |
✅ |
| SUFFIX |
✅ |
❌ |
❌ |
❌ |
✅ |
✅ |
| CONTAINS |
✅ |
❌ |
✅ |
❌ |
✅ |
✅ |
Error messages
pandas / narwhals-pandas:
TypeError: expected a string or tuple, not Expr
.venv/.../pandas/core/strings/accessor.py:3172: TypeError
narwhals-polars:
TypeError: cannot create expression literal for value of type Expr.
Hint: Pass `allow_object=True` to accept any value and create a literal of type Object.
Root cause
The starts_with, ends_with, and contains compile methods produce column-reference expressions for per-row pattern matching (e.g., matching a context value against each rule's string pattern). The polars and ibis backends handle column references natively in string operations, but:
- pandas passes the
Expr object directly to pd.Series.str.startswith() / .endswith() / .contains(), which expect scalar strings
- narwhals wraps pandas and inherits the same limitation; narwhals-polars fails on PREFIX/SUFFIX but not CONTAINS (suggesting partial support)
Reproduction
From mountainash-utils-rules (branch release/26.4.0):
hatch run test:test-perf-target "tests/test_benchmarks.py::TestStrategyIsolation" -v -k "pandas and PREFIX" --benchmark-only
Workaround
mountainash-utils-rules benchmarks skip these combinations:
_STRING_MATCH_STRATEGIES = {MatchStrategy.PREFIX, MatchStrategy.SUFFIX, MatchStrategy.CONTAINS}
_STRING_MATCH_BROKEN_BACKENDS = {"pandas", "narwhals-pandas", "narwhals-polars"}
Related
Summary
The
starts_with,ends_with, andcontainsexpression methods fail on pandas-based backends when used with per-row column references. The pandasstraccessor receives anExprobject where it expects a plain string.Discovered during cross-backend performance benchmarking of
mountainash-utils-rules.Affected backends and strategies
Error messages
pandas / narwhals-pandas:
narwhals-polars:
Root cause
The
starts_with,ends_with, andcontainscompile methods produce column-reference expressions for per-row pattern matching (e.g., matching a context value against each rule's string pattern). The polars and ibis backends handle column references natively in string operations, but:Exprobject directly topd.Series.str.startswith()/.endswith()/.contains(), which expect scalar stringsReproduction
From
mountainash-utils-rules(branchrelease/26.4.0):Workaround
mountainash-utils-rulesbenchmarks skip these combinations:Related
with_row_index(separate issue, same category of backend-specific gaps)