Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .github/config/mountainash_dependencies.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@
dependencies:
# - name: mountainash-auth-settings
# org-name: mountainash-io
- name: mountainash-constants
org-name: mountainash-io
# - name: mountainash-constants
# org-name: mountainash-io
- name: mountainash-data
org-name: mountainash-io
- name: mountainash-dataframes
- name: mountainash
org-name: mountainash-io
- name: mountainash-settings
org-name: mountainash-io
- name: mountainash-utils-dataclasses
org-name: mountainash-io
# - name: mountainash-utils-dataclasses
# org-name: mountainash-io
# - name: mountainash-utils-factoryclasses
# org-name: mountainash-io
# - name: mountainash-utils-files
Expand All @@ -26,5 +26,5 @@ dependencies:
# org-name: mountainash-io
# - name: mountainash-utils-rules
# org-name: mountainash-io
- name: mountainash-utils-ssh
org-name: mountainash-io
# - name: mountainash-utils-ssh
# org-name: mountainash-io
39 changes: 20 additions & 19 deletions .github/workflows/build-and-release-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ jobs:

- name: Python Dependencies
run: |
pip install hatchling==1.25.0
pip install hatch==1.14.2
pip install hatchling==1.29.0
pip install hatch==1.16.5

# Checkout Mountain Ash Dependencies
- name: Load Dependencies
Expand Down Expand Up @@ -378,51 +378,52 @@ jobs:
asset_name: ${{ env.PACKAGE_NAME }}-${{ env.VERSION }}-sbom-direct.json
asset_content_type: application/xml


- name: Setup Wheels Repository
run: |
# Configure git
git config --global user.name "GitHub Actions"
git config --global user.email "actions@github.com"

# Clone the wheels repository
git clone https://x-access-token:${{ secrets.CLONE_PRIVATE_REPOS_TOKEN }}@github.com/${{ env.ORGNAME }}/mountainash-wheels.git wheels-repo

# Go to the wheels repository
cd wheels-repo

# Generate a unique branch name using a timestamp
TIMESTAMP=$(date +%Y%m%d%H%M%S)
BRANCH_NAME="release/${{ env.PACKAGE_NAME }}-${{ env.VERSION }}-${TIMESTAMP}"

# Create a new branch for this release
git checkout -b $BRANCH_NAME

# Create package directory if it doesn't exist
mkdir -p ${{ env.PACKAGE_NAME }}

# Copy the newly built wheel to the repository
cp ../${{ steps.build.outputs.WHEEL_FILE }} ${{ env.PACKAGE_NAME }}/

# Add the new wheel file
git add .

# Commit the changes
git commit -m "Add ${{ steps.build.outputs.WHEEL_FILENAME }} to wheels repository"

# Push the branch to the repository
git push -u origin $BRANCH_NAME

# Export the branch name for later steps
echo "WHEELS_BRANCH=${BRANCH_NAME}" >> $GITHUB_ENV

- name: Create Pull Request
run: |
# Create a simpler PR body
PR_BODY="This PR adds the following wheel file to the wheels repository:\n- ${{ steps.build.outputs.WHEEL_FILENAME }}\n\nThis was automatically generated from the release workflow of ${{ github.repository }}."

# Properly escape the PR body for JSON
PR_BODY_ESCAPED=$(echo "$PR_BODY" | jq -Rs .)

# Create the PR
PR_RESPONSE=$(curl -X POST \
-H "Authorization: token ${{ secrets.CLONE_PRIVATE_REPOS_TOKEN }}" \
Expand All @@ -434,13 +435,13 @@ jobs:
\"head\": \"${WHEELS_BRANCH}\",
\"base\": \"main\"
}")

echo "API Response: $PR_RESPONSE"

# Extract PR URL and number
PR_URL=$(echo "$PR_RESPONSE" | jq -r '.html_url')
PR_NUMBER=$(echo "$PR_RESPONSE" | jq -r '.number')

# Add labels to the PR
if [ "$PR_NUMBER" != "null" ]; then
curl -X POST \
Expand All @@ -450,11 +451,11 @@ jobs:
-d '{
"labels": ["automated", "wheel"]
}'

echo "PR_URL=${PR_URL}" >> $GITHUB_ENV
echo "::notice::Pull Request created: ${PR_URL}"
else
echo "::error::Failed to create Pull Request"
echo "$PR_RESPONSE"
exit 1
fi
fi
21 changes: 19 additions & 2 deletions .github/workflows/main-release-build-dependencies.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ jobs:

- name: Python Dependencies
run: |
pip install hatchling==1.25.0
pip install hatch==1.14.2
pip install hatchling==1.29.0
pip install hatch==1.16.5

# Checkout Mountain Ash Dependencies
- name: Load Dependencies
Expand All @@ -65,6 +65,23 @@ jobs:
# ======================================================
# BUILD ARTIFACTS

- name: Debug Dep State
run: |
echo "== hatch.toml =="
cat hatch.toml
echo "== pyproject.toml =="
cat pyproject.toml
echo "== temp/ tree =="
ls -la temp/ || true
for d in temp/*/; do
echo "--- $d pyproject.toml ---"
grep -n "dataframes\|mountainash" "$d/pyproject.toml" || true
echo "--- $d hatch.toml ---"
grep -n "dataframes\|mountainash" "$d/hatch.toml" 2>/dev/null || true
done

- name: Setup Build Environment
env:
HATCH_VERBOSE: "2"
run: |
hatch env create ${{ env.BUILD_ENV }}
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ The rules engine supports 11 match strategies via the `MatchStrategy` enum, comp

**Backend support:**
- 9 strategies (EXACT, NOT_EQUAL, RANGE, GREATER_THAN, LESS_THAN, PREFIX, SUFFIX, CONTAINS, REGEX) compile cleanly on Polars, Ibis, and Narwhals backends — all support per-row patterns/thresholds via column references
- `SET_MEMBERSHIP` and `SET_EXCLUSION` currently use a Polars-native workaround (`ma.native(pl.col(...).list.contains(...))`) pending upstream `t_is_in`/`t_is_not_in` support for list-column references in mountainash-expressions
- `SET_MEMBERSHIP` and `SET_EXCLUSION` currently use a Polars-native workaround (`ma.native(pl.col(...).list.contains(...))`) pending upstream `t_is_in`/`t_is_not_in` support for list-column references in mountainash

**Unknown handling:** Sentinel values (`<NA>` for strings, `-999999999` for numerics) in either rule or context columns produce UNKNOWN (0) ternary results, which count as wildcards in ranking but do not eliminate the rule.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Given all of this - the rules engine and the mountainash expression system - wha

What you actually have

A coherent four-layer stack: data abstraction (mountainash-data/dataframes), cross-backend expressions (mountainash-expressions, Substrait-first), rules engines (mountainash-utils-rules), and a
A coherent four-layer stack: data abstraction (mountainash-data/dataframes), cross-backend expressions (mountainash, Substrait-first), rules engines (mountainash-utils-rules), and a
principles-driven governance model tying it all together. Most "frameworks" I see are loose collections of utilities that accrete around one person's work. Yours has an actual design philosophy, captured in
a principles repo with lettered categories, status markers, and explicit conflict-resolution rules. That level of architectural hygiene is rare — not just for solo/small-team projects, but for most
commercial projects too.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Replace the iterative dimension-by-dimension rule evaluation engine with a single-pass expression-based architecture using mountainash-expressions.
**Goal:** Replace the iterative dimension-by-dimension rule evaluation engine with a single-pass expression-based architecture using mountainash.

**Architecture:** Build ternary expression templates from dimension metadata at construction time, bind context values as literal columns at evaluation time, compile all dimensions in one `with_columns()` call. Survival = no FALSE(-1) in any dimension. Specificity = count of TRUE(1) values. Results ranked by specificity descending.

**Tech Stack:** mountainash-expressions (ternary logic, build-then-compile), polars (primary backend), ibis-framework (secondary), narwhals (tertiary), pydantic (models), pytest (testing)
**Tech Stack:** mountainash (ternary logic, build-then-compile), polars (primary backend), ibis-framework (secondary), narwhals (tertiary), pydantic (models), pytest (testing)

**Spec:** `docs/superpowers/specs/2026-04-03-expression-based-rules-engine-design.md`

Expand Down Expand Up @@ -47,7 +47,7 @@
| `tests/test_numpy_processor.py` | Delete | Old numpy tests |
| `tests/benchmarks/` | Delete | Old benchmark framework |
| `pyproject.toml` | Modify | Update dependencies |
| `hatch.toml` | Modify | Add mountainash-expressions dependency |
| `hatch.toml` | Modify | Add mountainash dependency |

---

Expand Down Expand Up @@ -121,13 +121,13 @@ Changes: removed `pandas>=2.2.0`, removed `sqlite` and `pandas` extras from ibis
In `hatch.toml`, add the mountainash expressions dependency to the `[envs.test]` dependencies list. Add this line alongside the other mountainash dependencies:

```
"mountainash @ {root:uri}/../mountainash-expressions",
"mountainash @ {root:uri}/../mountainash",
```

Do the same for `[envs.test_github]`:

```
"mountainash @ {root:uri}/temp/mountainash-expressions",
"mountainash @ {root:uri}/temp/mountainash",
```

- [ ] **Step 5: Commit**
Expand Down Expand Up @@ -1152,7 +1152,7 @@ Expected: FAIL — `ExpressionRulesEngine` does not exist (old engine.py is stil
- [ ] **Step 3: Implement engine.py**

```python
"""ExpressionRulesEngine: single-pass rule evaluation using mountainash-expressions."""
"""ExpressionRulesEngine: single-pass rule evaluation using mountainash."""

from __future__ import annotations

Expand All @@ -1171,7 +1171,7 @@ from mountainash_utils_rules.result import RuleResult


class ExpressionRulesEngine:
"""Rule evaluation engine using mountainash-expressions.
"""Rule evaluation engine using mountainash.

Compiles dimension metadata into expression templates at construction time,
then evaluates contexts against the rules DataFrame in a single-pass
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Extend `DimensionCompiler` from 3 to 11 match strategies and rewrite REGEX to be backend-agnostic using the now-consistent mountainash-expressions string API.
**Goal:** Extend `DimensionCompiler` from 3 to 11 match strategies and rewrite REGEX to be backend-agnostic using the now-consistent mountainash string API.

**Architecture:** Each new strategy is a small compile method in `DimensionCompiler`. String-returning operations (`starts_with`, `ends_with`, `contains`, `regex_contains`) share a `_compile_string_match` helper that wraps the boolean result in a sentinel-aware when/then ternary expression. Direct ternary ops (`t_eq`, `t_ne`, `t_gt`, `t_lt`, `t_is_in`, `t_is_not_in`) compile to one-liners using `t_col` with sentinel sets.

**Tech Stack:** mountainash-expressions (ternary logic, backend-agnostic string ops), polars (primary test backend), pydantic (Dimension model validation), pytest (testing)
**Tech Stack:** mountainash (ternary logic, backend-agnostic string ops), polars (primary test backend), pydantic (Dimension model validation), pytest (testing)

**Spec:** `docs/superpowers/specs/2026-04-07-extended-match-strategies-design.md`

**Prerequisite:** Upstream `mountainash-expressions` fixes (completed 2026-04-07):
**Prerequisite:** Upstream `mountainash` fixes (completed 2026-04-07):
- `contains`, `regex_contains`, `strpos`, `count_substring`, `like` accept column references
- `t_is_in`, `t_is_not_in` accept column references to list columns

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

**Architecture:** The dimension compiler is already backend-agnostic. This rewrite extends the same discipline to the engine pipeline (`with_columns`, `filter`, `sort`, `with_row_index`, `drop`) and to all `RuleResult` accessors (`count`, `best_match`, `explain`, `at_least`). All DataFrame operations go through `mountainash.relations.relation()` and `Relation` methods. All per-row operations go through `mountainash.expressions` (`ma.col`, `ma.lit`, `ma.least`, chained `.add()`).

**Tech Stack:** mountainash-expressions (relational + scalar APIs), mountainash-relations (Relation, count_rows, item, with_row_index), pydantic, pytest
**Tech Stack:** mountainash (relational + scalar APIs), mountainash-relations (Relation, count_rows, item, with_row_index), pydantic, pytest

**Spec:** `docs/superpowers/specs/2026-04-08-backend-agnostic-engine-and-result-design.md`

Expand Down Expand Up @@ -168,7 +168,7 @@ Read `src/mountainash_utils_rules/engine.py` in full to understand the current s
Replace `src/mountainash_utils_rules/engine.py` with this complete new content:

```python
"""ExpressionRulesEngine: single-pass rule evaluation using mountainash-expressions."""
"""ExpressionRulesEngine: single-pass rule evaluation using mountainash."""

from __future__ import annotations

Expand All @@ -189,7 +189,7 @@ from mountainash_utils_rules.result import RuleResult


class ExpressionRulesEngine:
"""Rule evaluation engine using mountainash-expressions.
"""Rule evaluation engine using mountainash.

Compiles dimension metadata into expression templates at construction time,
then evaluates contexts against the rules DataFrame in a single-pass
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

**Goal:** Parametrize the `mountainash-utils-rules` test suite across all 7 mountainash-supported DataFrame backends, replacing Polars-specific assertions with backend-agnostic reads via `mountainash.relations`.

**Architecture:** Rewrite `tests/conftest.py` to mirror the `mountainash-expressions` exemplar: pure-Python data fixtures, a `backend_name` param fixture over 7 backends, backend DataFrame factory fixtures, and a `basic_engine` that auto-parametrizes transitively. Test files replace direct Polars assertions with `mountainash.relations.relation(...).to_dict()`. SET_MEMBERSHIP / SET_EXCLUSION cases use strict `xfail` on non-Polars backends pointing at `mountainash-io/mountainash-expressions#75`.
**Architecture:** Rewrite `tests/conftest.py` to mirror the `mountainash` exemplar: pure-Python data fixtures, a `backend_name` param fixture over 7 backends, backend DataFrame factory fixtures, and a `basic_engine` that auto-parametrizes transitively. Test files replace direct Polars assertions with `mountainash.relations.relation(...).to_dict()`. SET_MEMBERSHIP / SET_EXCLUSION cases use strict `xfail` on non-Polars backends pointing at `mountainash-io/mountainash#75`.

**Tech Stack:** pytest, polars, pandas, narwhals, ibis-framework[duckdb,polars,sqlite], mountainash.relations, mountainash.expressions.

Expand Down Expand Up @@ -38,7 +38,7 @@ Write to `tests/conftest.py`:
```python
"""Shared fixtures for expression-based rules engine tests.

Mirrors the mountainash-expressions exemplar: data-as-dict fixtures + a
Mirrors the mountainash exemplar: data-as-dict fixtures + a
`backend_name` param fixture + per-backend DataFrame factory fixtures that
auto-parametrize every dependent test across all 7 supported backends.
"""
Expand Down Expand Up @@ -82,7 +82,7 @@ LIST_CAPABLE_BACKENDS = [

SET_MEMBERSHIP_XFAIL_REASON = (
"SET_MEMBERSHIP uses Polars-native workaround pending "
"mountainash-io/mountainash-expressions#75 (t_list_contains)"
"mountainash-io/mountainash#75 (t_list_contains)"
)


Expand Down Expand Up @@ -828,7 +828,7 @@ class TestMixedStrategyFraudDetection:
"""Exercises EXACT, SET_MEMBERSHIP, GREATER_THAN, and PREFIX together.

SET_MEMBERSHIP uses a Polars-native workaround (ma.native) pending
mountainash-io/mountainash-expressions#75. Non-Polars backends are
mountainash-io/mountainash#75. Non-Polars backends are
strict xfail — when #75 lands and the workaround is removed, these
flip XPASS and force removal of the markers.
"""
Expand Down Expand Up @@ -888,7 +888,7 @@ Expected: All non-SET tests pass × 7 backends; SET fraud tests are 3 PASS + 9 X
git add tests/test_integration.py
git commit -m "test(integration): parametrize fraud detection with xfail for non-polars SET

Refs mountainash-io/mountainash-expressions#75
Refs mountainash-io/mountainash#75

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>"
```
Expand Down Expand Up @@ -919,7 +919,7 @@ class TestBackendAgnosticism:
"""Smoke test: each strategy compiles and runs on every supported backend.

SET_MEMBERSHIP / SET_EXCLUSION are included but xfail-strict on non-Polars
backends, pending mountainash-io/mountainash-expressions#75.
backends, pending mountainash-io/mountainash#75.
"""

_SAMPLE_DATA = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**Date:** 2026-04-03
**Status:** Approved
**Scope:** Complete rearchitecture of mountainash-utils-rules to use mountainash-expressions
**Scope:** Complete rearchitecture of mountainash-utils-rules to use mountainash

## Summary

Expand All @@ -13,7 +13,7 @@ This is a clean break — all existing engines (`RulesEngine`, `HybridRulesEngin
## Goals

1. **Eliminate iterative evaluation** — current engine applies ~10 mutate() calls per dimension; new engine evaluates all dimensions in a single `with_columns()` call
2. **Leverage mountainash-expressions** — build-then-compile pattern, ternary logic, backend agnosticism
2. **Leverage mountainash** — build-then-compile pattern, ternary logic, backend agnosticism
3. **Backend-agnostic** — same engine works with Polars, Ibis, and Narwhals DataFrames; test primarily with Polars
4. **Dual API** — convenience path (DataFrame + dimension metadata) and advanced path (raw expressions)
5. **Built-in observability** — per-dimension ternary columns in results, no separate observer infrastructure
Expand Down Expand Up @@ -267,7 +267,7 @@ The `__` prefix prevents collision with rule columns. `__t_*` columns are omitte

### Ternary Logic Mapping

The current prime-based system (2/3/5) is replaced by mountainash-expressions' integer sentinels:
The current prime-based system (2/3/5) is replaced by mountainash' integer sentinels:

| Concept | Old (prime) | New (expressions) |
|---------|-------------|-------------------|
Expand Down Expand Up @@ -378,7 +378,7 @@ tests/
## Dependencies

**Added:**
- `mountainash-expressions` — core expression library
- `mountainash` — core expression library

**Retained:**
- `polars` — primary test backend
Expand Down
Loading
Loading