Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
236 changes: 236 additions & 0 deletions docs/superpowers/plans/2026-05-13-profile-spec-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
# mountainash-data: profile-spec migration plan

**Goal:** Migrate `mountainash-data` to the new `ProfileSpec` / `Profile` vocabulary introduced in `mountainash-settings 26.5.0`. Follow the [migration guide](../../../../mountainash-settings/docs/superpowers/specs/2026-05-13-profile-spec-rename-design.md#migration-guide-for-downstream-consumers) verbatim.

**Architecture:** Mechanical search-and-replace across 21 backend files plus updates to `descriptor.py`, `registry.py`, `profile.py`, and test files. Add a PEP 562 `__getattr__` shim in `mountainash-data`'s own `descriptor.py` so any downstream consumer of `mountainash-data` gets the same one-release deprecation window.

**Tech Stack:** Python 3.10+, pydantic 2.x, mountainash-settings ≥26.5.0 (via path-based dep), pytest, hatch.

**Upstream spec:** `../../../../mountainash-settings/docs/superpowers/specs/2026-05-13-profile-spec-rename-design.md`

**Out of scope:**
- Renaming `*AuthSettings` concrete class names (e.g. `PostgreSQLAuthSettings`) — explicitly deferred in the upstream spec.
- Adding a `mountainash-settings` version pin — `mountainash-data` uses a path-based dependency (`{root:uri}/../mountainash-settings`), so version coordination is implicit.

---

## File survey (from `grep -ln`)

**Source files using old names (21):**

```
src/mountainash_data/core/settings/registry.py
src/mountainash_data/core/settings/profile.py
src/mountainash_data/core/settings/descriptor.py
src/mountainash_data/core/settings/sqlite.py
src/mountainash_data/core/settings/duckdb.py (likely)
src/mountainash_data/core/settings/postgresql.py
src/mountainash_data/core/settings/mysql.py
src/mountainash_data/core/settings/mssql.py
src/mountainash_data/core/settings/snowflake.py
src/mountainash_data/core/settings/redshift.py
src/mountainash_data/core/settings/bigquery.py (likely)
src/mountainash_data/core/settings/databricks.py (likely)
src/mountainash_data/core/settings/motherduck.py
src/mountainash_data/core/settings/clickhouse.py (likely)
src/mountainash_data/core/settings/trino.py
src/mountainash_data/core/settings/singlestoredb.py
src/mountainash_data/core/settings/exasol.py
src/mountainash_data/core/settings/impala.py
src/mountainash_data/core/settings/materialize.py
src/mountainash_data/core/settings/risingwave.py
src/mountainash_data/core/settings/druid.py (likely)
src/mountainash_data/core/settings/pyspark.py
src/mountainash_data/core/settings/pyiceberg_rest.py
```

**Test files using old names (3):**

```
tests/test_unit/core/settings/test_descriptor.py
tests/test_unit/core/settings/test_profile.py
tests/test_unit/core/settings/test_descriptors_invariants.py
```

---

## Rename table

Apply to every file touched:

| Old | New |
|---|---|
| `from mountainash_settings.profiles import ProfileDescriptor` | `from mountainash_settings.profiles import ProfileSpec` |
| `from mountainash_settings.profiles.descriptor import _Missing` | `from mountainash_settings.profiles import Missing` |
| `class BackendDescriptor(ProfileDescriptor)` | `class BackendSpec(ProfileSpec)` |
| Any `BackendDescriptor` reference | `BackendSpec` |
| `*_DESCRIPTOR = BackendDescriptor(...)` | `*_SPEC = BackendSpec(...)` |
| Every reference to `POSTGRESQL_DESCRIPTOR` etc. | `POSTGRESQL_SPEC` etc. |
| `@register(POSTGRESQL_DESCRIPTOR)` | `@register` (argument-free) |
| `__descriptor__ = POSTGRESQL_DESCRIPTOR` | `__spec__ = POSTGRESQL_SPEC` |
| `Registry("databases")` | `Registry("databases", spec_type=BackendSpec, profile_type=ConnectionProfile)` |
| Local MRO walk in `to_driver_kwargs` | `lookup_class_var` import from `mountainash_settings` |
| `descriptor_invariants_for` | `spec_invariants_for` |
| `TestDescriptorInvariants_*` | `TestSpecInvariants_*` (in expected pytest output assertions) |

---

## Tasks

### Task A: `descriptor.py` rename + shim

**Files:**
- Modify: `src/mountainash_data/core/settings/descriptor.py`
- Modify: `src/mountainash_data/core/settings/__init__.py` (if `BackendDescriptor` is re-exported there)

**Required changes:**

1. Rename `class BackendDescriptor` → `class BackendSpec`.
2. Replace `from mountainash_settings.profiles.descriptor import _Missing` with `from mountainash_settings.profiles import Missing`.
3. Replace `from mountainash_settings.profiles import ProfileDescriptor` with `from mountainash_settings.profiles import ProfileSpec`. Update `class BackendSpec(ProfileSpec)`.
4. Update `__all__` to use `BackendSpec` and `Missing`.
5. Add PEP 562 `__getattr__` shim at the bottom of `descriptor.py`:

```python
import warnings


_DEPRECATED = {
"BackendDescriptor": ("BackendSpec", BackendSpec),
"_Missing": ("Missing", Missing),
}


def __getattr__(name):
if name in _DEPRECATED:
new_name, obj = _DEPRECATED[name]
warnings.warn(
f"{name!r} is renamed to {new_name!r} in mountainash-data. "
f"Update imports to use the new name.",
DeprecationWarning, stacklevel=2,
)
return obj
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
```

6. Update `__init__.py` re-exports if `BackendDescriptor` was previously exported — change to `BackendSpec` and add a top-level shim if downstream consumers may import directly from the package root.

### Task B: `registry.py` constraints + `profile.py` lookup helper

**Files:**
- Modify: `src/mountainash_data/core/settings/registry.py`
- Modify: `src/mountainash_data/core/settings/profile.py`

**registry.py changes:**

1. Update the `DATABASES_REGISTRY` construction:

```python
# Before
DATABASES_REGISTRY = Registry("databases")

# After
DATABASES_REGISTRY = Registry(
"databases",
spec_type=BackendSpec,
profile_type=ConnectionProfile,
)
```

2. Update any `descriptor_invariants_for` references to `spec_invariants_for`.

**profile.py changes:**

3. Replace local MRO walk in `to_driver_kwargs()`:

```python
# Before — local MRO walk
adapter = type(self).__dict__.get("__adapter__")
if adapter is None:
for base in type(self).__mro__[1:]:
candidate = base.__dict__.get("__adapter__")
if candidate is not None:
adapter = candidate
break

# After — public helper
from mountainash_settings import lookup_class_var
adapter = lookup_class_var(type(self), "__adapter__")
```

The import can go at the top of the file rather than inline.

### Task C: 21 backend file sweep

For each file in `src/mountainash_data/core/settings/` matching `^(?!__init__|descriptor|profile|registry).*\.py$`:

**Replace:**
- `BackendDescriptor` → `BackendSpec` (imports and constructor calls)
- `<UPPER>_DESCRIPTOR = BackendDescriptor(` → `<UPPER>_SPEC = BackendSpec(`
- Every reference to `<UPPER>_DESCRIPTOR` → `<UPPER>_SPEC`
- `@register(<UPPER>_DESCRIPTOR)` → `@register` (drop the argument)
- `__descriptor__ = <UPPER>_DESCRIPTOR` (or whatever spec it points at) → `__spec__ = <UPPER>_SPEC`

Each file is independent. Apply the same mechanical pattern. Verify after each that imports resolve.

### Task D: Tests

**Files:**
- Modify: `tests/test_unit/core/settings/test_profile.py`
- Modify: `tests/test_unit/core/settings/test_descriptor.py` (or rename to `test_spec.py` if desired — optional)
- Modify: `tests/test_unit/core/settings/test_descriptors_invariants.py` (consider renaming to `test_spec_invariants.py`)

Apply the same rename table. Update any references to the old API names.

### Task E: Version bump + verification

**Files:**
- Modify: `src/mountainash_data/__version__.py`

**Steps:**

1. Bump version. Current is `2026.04.2`. Following `mountainash-data`'s CalVer pattern (`YYYY.MM.MICRO`), the next release is `2026.05.0` (since we're in May).

2. Run full test suite:
```bash
hatch run test:test
```

3. Run with deprecation warnings escalated to errors:
```bash
hatch run test:test -W "error::DeprecationWarning" -W "default::DeprecationWarning:mountainash_data.core.settings.descriptor"
```
The `-W default` filter explicitly allows warnings from `mountainash-data`'s own descriptor shim (which exists by design). Any warning from elsewhere fails the run — that's the migration completion check.

4. Run lint:
```bash
hatch run ruff:check
```

5. Build:
```bash
hatch build
```

### Task F: Push and PR

1. Push the feature branch.
2. Open PR targeting `develop` with the same level of detail as the upstream PR (description, test plan, removal commitment).

### Restore stashed working tree

After the PR is open:

```bash
git stash pop # restore hatch.toml reorder + .claude/worktrees/settings-registry
```

(Or leave for the user to handle.)

---

## Execution strategy

The work is mechanical. Single-subagent dispatch can handle Tasks A-D as one batch since they're all in the same package and the transformations are templated. Task E and F can be done manually.

This plan does not list individual TDD steps because the upstream contract guarantees behaviour: the new names resolve to the same objects as the old names (via deprecation aliases). Running the existing test suite is the verification. New tests are not added in this PR — the upstream PR added all the deprecation tests; this PR is purely a consumer migration.
2 changes: 1 addition & 1 deletion src/mountainash_data/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__="2026.04.2"
__version__="2026.05.0"
6 changes: 3 additions & 3 deletions src/mountainash_data/backends/ibis/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,16 +227,16 @@ def _init_from_settings(
obj_settings = settings_params.settings_class.get_settings(
settings_parameters=settings_params
)
descriptor = getattr(obj_settings, "__descriptor__", None)
descriptor = getattr(obj_settings, "__spec__", None)
if descriptor is None or getattr(descriptor, "ibis_dialect", None) is None:
raise ValueError(
f"Settings class {type(obj_settings).__name__} has no "
f"ibis_dialect on its descriptor"
f"ibis_dialect on its spec"
)
resolved_dialect = descriptor.ibis_dialect
if resolved_dialect not in DIALECTS:
raise KeyError(
f"Unknown ibis dialect {resolved_dialect!r} from descriptor. "
f"Unknown ibis dialect {resolved_dialect!r} from spec. "
f"Available: {sorted(DIALECTS)}"
)
driver_kwargs = obj_settings.to_driver_kwargs()
Expand Down
29 changes: 25 additions & 4 deletions src/mountainash_data/core/settings/__init__.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
"""Backend settings — declarative descriptor + registry.
"""Backend settings — declarative spec + registry.

The *AuthSettings classes below are stable import anchors; internally each
class body is a two-line shell (``__descriptor__`` + ``__adapter__``).
class body is a two-line shell (``__spec__`` + ``__adapter__``).
"""

from __future__ import annotations

# Core primitives
from .descriptor import MISSING, BackendDescriptor, ParameterSpec
from .descriptor import MISSING, Missing, BackendSpec, ParameterSpec
from .profile import ConnectionProfile
from .registry import (
DATABASES_REGISTRY,
Expand Down Expand Up @@ -55,9 +55,30 @@ class body is a two-line shell (``__descriptor__`` + ``__adapter__``).
from .druid import DruidAuthSettings
from .pyiceberg_rest import PyIcebergRestAuthSettings

import warnings as _warnings


_DEPRECATED_PKG = {
"BackendDescriptor": ("BackendSpec", BackendSpec),
"_Missing": ("Missing", Missing),
}


def __getattr__(name: str):
if name in _DEPRECATED_PKG:
new_name, obj = _DEPRECATED_PKG[name]
_warnings.warn(
f"{name!r} is renamed to {new_name!r} in mountainash-data. "
f"Update imports to use the new name.",
DeprecationWarning, stacklevel=2,
)
return obj
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")


__all__ = [
# primitives
"MISSING", "BackendDescriptor", "ParameterSpec", "ConnectionProfile",
"MISSING", "Missing", "BackendSpec", "ParameterSpec", "ConnectionProfile",
"DATABASES_REGISTRY", "REGISTRY",
"get_descriptor", "get_settings_class", "register",
# auth
Expand Down
10 changes: 5 additions & 5 deletions src/mountainash_data/core/settings/bigquery.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
from ..constants import CONST_DB_PROVIDER_TYPE
from .adapters import bigquery as _adapter
from mountainash_settings.auth import NoAuth, ServiceAccountAuth
from .descriptor import BackendDescriptor, ParameterSpec
from .descriptor import BackendSpec, ParameterSpec
from .profile import ConnectionProfile
from .registry import register

__all__ = ["BigQueryAuthSettings", "BIGQUERY_DESCRIPTOR"]
__all__ = ["BigQueryAuthSettings", "BIGQUERY_SPEC"]

_PROJECT_ID_RE = re.compile(r"^[a-z][a-z0-9-]{4,28}[a-z0-9]$")

Expand All @@ -32,7 +32,7 @@ def _validate_project_id(value: str) -> str:
return value


BIGQUERY_DESCRIPTOR = BackendDescriptor(
BIGQUERY_SPEC = BackendSpec(
name="bigquery",
provider_type=CONST_DB_PROVIDER_TYPE.BIGQUERY,
connection_string_scheme="bigquery://",
Expand Down Expand Up @@ -60,9 +60,9 @@ def _validate_project_id(value: str) -> str:
)


@register(BIGQUERY_DESCRIPTOR)
@register
class BigQueryAuthSettings(ConnectionProfile):
__descriptor__ = BIGQUERY_DESCRIPTOR
__spec__ = BIGQUERY_SPEC
__adapter__ = staticmethod(_adapter.build_driver_kwargs)

@field_validator("PROJECT_ID", check_fields=False)
Expand Down
8 changes: 4 additions & 4 deletions src/mountainash_data/core/settings/clickhouse.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@

from ..constants import CONST_DB_PROVIDER_TYPE
from mountainash_settings.auth import NoAuth, PasswordAuth
from .descriptor import BackendDescriptor, ParameterSpec
from .descriptor import BackendSpec, ParameterSpec
from .profile import ConnectionProfile
from .registry import register


CLICKHOUSE_DESCRIPTOR = BackendDescriptor(
CLICKHOUSE_SPEC = BackendSpec(
name="clickhouse",
provider_type=CONST_DB_PROVIDER_TYPE.CLICKHOUSE,
default_port=9000,
Expand Down Expand Up @@ -44,6 +44,6 @@
)


@register(CLICKHOUSE_DESCRIPTOR)
@register
class ClickHouseAuthSettings(ConnectionProfile):
__descriptor__ = CLICKHOUSE_DESCRIPTOR
__spec__ = CLICKHOUSE_SPEC
8 changes: 4 additions & 4 deletions src/mountainash_data/core/settings/databricks.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@
from ..constants import CONST_DB_PROVIDER_TYPE
from .adapters import databricks as _adapter
from mountainash_settings.auth import NoAuth, PasswordAuth, TokenAuth
from .descriptor import BackendDescriptor, ParameterSpec
from .descriptor import BackendSpec, ParameterSpec
from .profile import ConnectionProfile
from .registry import register


DATABRICKS_DESCRIPTOR = BackendDescriptor(
DATABRICKS_SPEC = BackendSpec(
name="databricks",
provider_type=CONST_DB_PROVIDER_TYPE.DATABRICKS,
ibis_dialect="databricks",
Expand All @@ -37,7 +37,7 @@
)


@register(DATABRICKS_DESCRIPTOR)
@register
class DatabricksAuthSettings(ConnectionProfile):
__descriptor__ = DATABRICKS_DESCRIPTOR
__spec__ = DATABRICKS_SPEC
__adapter__ = staticmethod(_adapter.build_driver_kwargs)
Loading
Loading