Suhail 2026 source expansion (+ track suhail_pipeline package)

.gitignore

@@ -200,3 +200,4 @@ docs/reports/generated/*.md
 # Allow all files in sample_data to be tracked, even if normally ignored
 !/sample_data/
 !/sample_data/**
+.gstack/

README.md

@@ -1,6 +1,6 @@
 [![CI](https://github.com/<your-org-or-username>/<your-repo>/actions/workflows/ci.yml/badge.svg)](https://github.com/<your-org-or-username>/<your-repo>/actions/workflows/ci.yml)
 
-# Meshic Geospatial Data Pipeline
+# Suhail Geospatial Data Pipeline
 
 A sophisticated **two-stage geospatial data processing pipeline** for the Saudi real estate market. The pipeline is now fully database-driven: all tiles to be processed are stored in the `tile_urls` table, supporting province-wide and all-Saudi scrapes with resumable, robust processing.
 
@@ -102,29 +102,29 @@ cp .env.example .env
 #### Stage 1: Geometric Processing (DB-driven)
 ```bash
 # Seed tiles (optional; per province or all provinces)
-meshic-pipeline seed-tiles --province riyadh --limit 1000 --stride 2
+suhail-pipeline seed-tiles --province riyadh --limit 1000 --stride 2
 
 # Process tiles using the DB queue (recommended)
-meshic-pipeline db-geometric --batch-size 1000 --concurrency 5 --adaptive
+suhail-pipeline db-geometric --batch-size 1000 --concurrency 5 --adaptive
 
 # Or traditional bbox mode
-meshic-pipeline geometric --bbox 46.428223 24.367114 47.010498 24.896402
+suhail-pipeline geometric --bbox 46.428223 24.367114 47.010498 24.896402
 ```
 
 #### Stage 2: Enrichment
 ```bash
 # Fast enrichment (new parcels that need data)
-meshic-pipeline fast-enrich --batch-size 200
+suhail-pipeline fast-enrich --batch-size 200
 
 # Incremental enrichment (stale by days)
-meshic-pipeline incremental-enrich --days-old 30 --batch-size 100
+suhail-pipeline incremental-enrich --days-old 30 --batch-size 100
 
 # Delta enrichment (only parcels with price changes)
-meshic-pipeline delta-enrich --auto-geometric
+suhail-pipeline delta-enrich --auto-geometric
 
 # Monitoring
-meshic-pipeline monitor status
-meshic-pipeline monitor recommend
+suhail-pipeline monitor status
+suhail-pipeline monitor recommend
 ```
 
 ## 🔄 Enrichment Strategies
@@ -136,7 +136,7 @@ The pipeline provides **multiple enrichment modes** to leverage this insight:
 
 ### **🎯 TRIGGER-BASED** (Maximum Efficiency)
 ```bash
-meshic-pipeline enrich fast-enrich --batch-size 400
+suhail-pipeline enrich fast-enrich --batch-size 400
 ```
  - **🚀 Leverages your insight**: Only processes parcels with `transaction_price > 0`
 - **93.3% efficiency gain**: Skips 962,796 parcels that don't need enrichment
@@ -145,7 +145,7 @@ meshic-pipeline enrich fast-enrich --batch-size 400
 
 ### **🆕 NEW PARCELS** (Standard Approach)
 ```bash
-meshic-pipeline enrich fast-enrich --batch-size 200
+suhail-pipeline enrich fast-enrich --batch-size 200
 ```
 - Processes parcels never enriched before (same as trigger-based but different implementation)
 - Perfect for initial runs or capturing new parcels
@@ -154,18 +154,18 @@ meshic-pipeline enrich fast-enrich --batch-size 200
 ### **🔄 INCREMENTAL UPDATES** (Weekly/Monthly) 
 ```bash
 # Weekly updates (recommended)
-meshic-pipeline enrich incremental-enrich --days-old 7 --batch-size 100
+suhail-pipeline enrich incremental-enrich --days-old 7 --batch-size 100
 
 # Monthly updates
-meshic-pipeline enrich incremental-enrich --days-old 30 --batch-size 100
+suhail-pipeline enrich incremental-enrich --days-old 30 --batch-size 100
 ```
 - **🎯 Captures new transactions on existing parcels**
 - Re-processes parcels not enriched recently
 - **Use case**: Ongoing operations to catch new transaction data
 
 ### **🔥 FULL REFRESH** (Quarterly)
 ```bash
-meshic-pipeline enrich full-refresh --batch-size 50
+suhail-pipeline enrich full-refresh --batch-size 50
 ```
 - Re-processes ALL enrichable parcels
 - Guarantees 100% data completeness
@@ -174,13 +174,13 @@ meshic-pipeline enrich full-refresh --batch-size 50
 ### **🎯 DELTA ENRICHMENT** (Revolutionary Precision) 
 ```bash
 # Automatic workflow (recommended)
-meshic-pipeline enrich delta-enrich --auto-geometric
+suhail-pipeline enrich delta-enrich --auto-geometric
 
 # Manual workflow (if fresh MVT data already exists)
-meshic-pipeline enrich delta-enrich
+suhail-pipeline enrich delta-enrich
 
 # Testing with limits
-meshic-pipeline enrich delta-enrich --limit 100 --auto-geometric
+suhail-pipeline enrich delta-enrich --limit 100 --auto-geometric
 ```
 - **🚀 MVT-based change detection**: Only enriches parcels with actual transaction price changes
 - **Perfect precision**: No false positives from time-based approaches
@@ -197,19 +197,19 @@ meshic-pipeline enrich delta-enrich --limit 100 --auto-geometric
 ### Status Monitoring (CLI)
 ```bash
 # Queue status, enrichment coverage, failures
-meshic-pipeline monitor status
+suhail-pipeline monitor status
 
 # Automated recommendations
-meshic-pipeline monitor recommend
+suhail-pipeline monitor recommend
 
 # Scheduling guidance
-meshic-pipeline monitor schedule-info
+suhail-pipeline monitor schedule-info
 
 # Reset stale in_progress tiles (for cron)
-meshic-pipeline monitor reset-stale -- --stale-minutes 60
+suhail-pipeline monitor reset-stale -- --stale-minutes 60
 
 # Sample performance measurements (writes docs/reports)
-meshic-pipeline monitor perf -- --label baseline --iterations 5
+suhail-pipeline monitor perf -- --label baseline --iterations 5
 
 # Repair missing province metadata (tile URL/bbox)
 python scripts/util/backfill_province_metadata.py --province-id 21012
@@ -256,12 +256,12 @@ layers: [parcels, transactions, neighborhoods, ...]
 ### **Performance Tuning**
 ```bash
 # High-performance settings
-meshic-pipeline enrich incremental-enrich \
+suhail-pipeline enrich incremental-enrich \
   --batch-size 500 \
   --days-old 7
 
 # Memory-optimized settings
-meshic-pipeline enrich incremental-enrich \
+suhail-pipeline enrich incremental-enrich \
   --batch-size 100 \
   --days-old 7
 ```
@@ -294,16 +294,16 @@ The pipeline **guarantees** capture of new transactions through:
 ## 🚨 Important Notes
 
 ### **For New Deployments**
-1. Seed provinces (optional): `meshic-pipeline seed-tiles --province riyadh`
-2. Run geometric pipeline: `meshic-pipeline db-geometric`
-3. Run initial enrichment: `meshic-pipeline fast-enrich`
-4. Setup monitoring: `meshic-pipeline monitor status`
+1. Seed provinces (optional): `suhail-pipeline seed-tiles --province riyadh`
+2. Run geometric pipeline: `suhail-pipeline db-geometric`
+3. Run initial enrichment: `suhail-pipeline fast-enrich`
+4. Setup monitoring: `suhail-pipeline monitor status`
 
 ### **For Ongoing Operations**
 - **💡 LEVERAGE THE INSIGHT**: Use `fast-enrich` after geometric pipeline for maximum efficiency
 - **Never use only `fast-enrich`** for ongoing operations - it misses new transactions on existing parcels
 - **Use `incremental-enrich`** weekly to capture new transaction data
-- **Monitor regularly** with `uv run meshic-pipeline monitor recommend`
+- **Monitor regularly** with `uv run suhail-pipeline monitor recommend`
 
 ### **🚀 EFFICIENCY BREAKTHROUGH**
 Your insight reveals a **93.3% efficiency gain**:
@@ -335,10 +335,10 @@ uv run pytest tests/integration
 check_db
 
 # Memory issues during processing
-meshic-pipeline incremental-enrich --batch-size 50
+suhail-pipeline incremental-enrich --batch-size 50
 
 # Check enrichment status
-meshic-pipeline monitor status
+suhail-pipeline monitor status
 ```
 
 ### **Performance Optimization**
@@ -377,7 +377,7 @@ Welcome! If you're picking up the pipeline/data debugging and remediation, here
 If you see `ModuleNotFoundError: No module named 'src'` when running enrichment via the CLI, use this workaround:
 
 ```bash
-PYTHONPATH=$(pwd) python src/meshic_pipeline/run_enrichment_pipeline.py fast-enrich --limit 100
+PYTHONPATH=$(pwd) python src/suhail_pipeline/run_enrichment_pipeline.py fast-enrich --limit 100
 ```
 
 This is required because the CLI currently invokes the enrichment script as a subprocess, which does not set up the Python path correctly. This will be fixed in a future release.
@@ -419,33 +419,33 @@ This does not affect current functionality but should be addressed in the future
 
 ### Core Commands
 
-- `meshic-pipeline geometric [--bbox ...] [--recreate-db] [--save-as-temp ...]`
+- `suhail-pipeline geometric [--bbox ...] [--recreate-db] [--save-as-temp ...]`
   - Run geometric pipeline (Stage 1)
   - Options:
     - `--bbox min_lon min_lat max_lon max_lat` — Bounding box for processing
     - `--recreate-db` — Drop and recreate the database schema
     - `--save-as-temp <table>` — Save parcels to a temporary table
 
-- `meshic-pipeline fast-enrich [--batch-size ...] [--limit ...]`
+- `suhail-pipeline fast-enrich [--batch-size ...] [--limit ...]`
   - Enrich new parcels with transaction prices
   - Options:
     - `--batch-size <int>` — Number of parcels per batch (default: 200)
     - `--limit <int>` — Limit parcels for testing
 
-- `meshic-pipeline incremental-enrich [--batch-size ...] [--days-old ...] [--limit ...]`
+- `suhail-pipeline incremental-enrich [--batch-size ...] [--days-old ...] [--limit ...]`
   - Enrich parcels not updated in X days
   - Options:
     - `--batch-size <int>` — Number of parcels per batch (default: 100)
     - `--days-old <int>` — Days old threshold (default: 30)
     - `--limit <int>` — Limit parcels for testing
 
-- `meshic-pipeline full-refresh [--batch-size ...] [--limit ...]`
+- `suhail-pipeline full-refresh [--batch-size ...] [--limit ...]`
   - Enrich ALL parcels (complete refresh)
   - Options:
     - `--batch-size <int>` — Number of parcels per batch (default: 50)
     - `--limit <int>` — Limit parcels for testing
 
-- `meshic-pipeline delta-enrich [--batch-size ...] [--limit ...] [--fresh-table ...] [--auto-geometric] [--show-details/--no-details]`
+- `suhail-pipeline delta-enrich [--batch-size ...] [--limit ...] [--fresh-table ...] [--auto-geometric] [--show-details/--no-details]`
   - Only process parcels with actual transaction price changes
   - Options:
     - `--batch-size <int>` — Number of parcels per batch (default: 200)
@@ -456,67 +456,67 @@ This does not affect current functionality but should be addressed in the future
 
 ### Advanced/Composite Commands
 
-- `meshic-pipeline smart-pipeline [--geometric-first] [--batch-size ...] [--bbox ...]`
+- `suhail-pipeline smart-pipeline [--geometric-first] [--batch-size ...] [--bbox ...]`
   - Complete geometric + enrichment workflow (recommended for full runs)
 
-- `meshic-pipeline monitor <status|recommend|schedule-info>`
+- `suhail-pipeline monitor <status|recommend|schedule-info>`
   - Run enrichment monitoring commands
 
-- `meshic-pipeline province-geometric <province> [--strategy ...] [--recreate-db] [--save-as-temp ...]`
+- `suhail-pipeline province-geometric <province> [--strategy ...] [--recreate-db] [--save-as-temp ...]`
   - Geometric pipeline for a specific province
   - Options:
     - `province` — Province name (al_qassim, riyadh, madinah, asir, eastern, makkah)
     - `--strategy <str>` — Discovery strategy (optimal, efficient, comprehensive; default: optimal)
     - `--recreate-db` — Drop and recreate the database schema
     - `--save-as-temp <table>` — Save parcels to a temporary table
 
-- `meshic-pipeline saudi-arabia-geometric [--strategy ...] [--recreate-db] [--save-as-temp ...]`
+- `suhail-pipeline saudi-arabia-geometric [--strategy ...] [--recreate-db] [--save-as-temp ...]`
   - Geometric pipeline for ALL Saudi provinces
 
-- `meshic-pipeline discovery-summary`
+- `suhail-pipeline discovery-summary`
   - Show province discovery capabilities/statistics
 
-- `meshic-pipeline province-pipeline <province> [--strategy ...] [--batch-size ...] [--geometric-first]`
+- `suhail-pipeline province-pipeline <province> [--strategy ...] [--batch-size ...] [--geometric-first]`
   - Complete province pipeline: geometric + enrichment for specific province
 
-- `meshic-pipeline saudi-pipeline [--strategy ...] [--batch-size ...] [--geometric-first]`
+- `suhail-pipeline saudi-pipeline [--strategy ...] [--batch-size ...] [--geometric-first]`
   - Complete Saudi Arabia pipeline: ALL provinces geometric + enrichment
 
 ### Usage Examples
 
 ```bash
 # Run geometric pipeline for a bounding box
-meshic-pipeline geometric --bbox 46.428223 24.367114 47.010498 24.896402
+suhail-pipeline geometric --bbox 46.428223 24.367114 47.010498 24.896402
 
 # Enrich new parcels (fast)
-meshic-pipeline fast-enrich --batch-size 400
+suhail-pipeline fast-enrich --batch-size 400
 
 # Incremental enrichment (parcels not updated in 7 days)
-meshic-pipeline incremental-enrich --days-old 7 --batch-size 100
+suhail-pipeline incremental-enrich --days-old 7 --batch-size 100
 
 # Full refresh (all enrichable parcels)
-meshic-pipeline full-refresh --batch-size 50
+suhail-pipeline full-refresh --batch-size 50
 
 # Delta enrichment (only parcels with price changes, auto-run geometric)
-meshic-pipeline delta-enrich --auto-geometric
+suhail-pipeline delta-enrich --auto-geometric
 
 # Province-wide geometric processing
-meshic-pipeline province-geometric riyadh --strategy optimal
+suhail-pipeline province-geometric riyadh --strategy optimal
 
 # All-province geometric processing
-meshic-pipeline saudi-arabia-geometric --strategy efficient
+suhail-pipeline saudi-arabia-geometric --strategy efficient
 
 # Complete province pipeline (geometric + enrichment)
-meshic-pipeline province-pipeline riyadh --strategy optimal --batch-size 300
+suhail-pipeline province-pipeline riyadh --strategy optimal --batch-size 300
 
 # Complete Saudi pipeline (all provinces)
-meshic-pipeline saudi-pipeline --strategy efficient --batch-size 500
+suhail-pipeline saudi-pipeline --strategy efficient --batch-size 500
 
 # Show discovery summary
-meshic-pipeline discovery-summary
+suhail-pipeline discovery-summary
 
 # Monitor enrichment status
-meshic-pipeline monitor status
+suhail-pipeline monitor status
 ```
 
 ## 🧪 Running Tests

REPOSITORY_INDEX.md

@@ -6,7 +6,7 @@ High-level map of the whole tree for navigation and onboarding. Pair with [docs/
 
 | Path | Role |
 |------|------|
-| [src/meshic_pipeline/](src/meshic_pipeline/) | Installable package: CLI (`cli.py`), geometric/enrichment runners, persistence, enrichment, geometry, decoder, discovery. |
+| [src/suhail_pipeline/](src/suhail_pipeline/) | Installable package: CLI (`cli.py`), geometric/enrichment runners, persistence, enrichment, geometry, decoder, discovery. |
 | [alembic/](alembic/) | Database migrations (`env.py`, `versions/`, `versions_backup/`). |
 | [tests/](tests/) | `pytest` suites: `unit/`, `integration/`. |
 | [scripts/](scripts/) | Operational and utility scripts (`util/`, `db/`, reports). |
@@ -15,7 +15,7 @@ High-level map of the whole tree for navigation and onboarding. Pair with [docs/
 
 | Path | Role |
 |------|------|
-| [pyproject.toml](pyproject.toml) | Project `meshic-pipeline`, dependencies, `[project.scripts]`, pytest config, `[dependency-groups].dev`. |
+| [pyproject.toml](pyproject.toml) | Project `suhail-pipeline`, dependencies, `[project.scripts]`, pytest config, `[dependency-groups].dev`. |
 | [uv.lock](uv.lock) | Locked dependency graph for reproducible installs (`uv sync --frozen`). |
 | [.github/workflows/ci.yml](.github/workflows/ci.yml) | CI: `astral-sh/setup-uv`, `uv sync --all-groups --frozen`, `uv run pytest`. |
 
@@ -57,5 +57,5 @@ High-level map of the whole tree for navigation and onboarding. Pair with [docs/
 
 ## Entrypoints (from `pyproject.toml`)
 
-- `meshic-pipeline` → `meshic_pipeline.cli:app`
-- `check_db` → `meshic_pipeline.utils.db_checker:app`
+- `suhail-pipeline` → `suhail_pipeline.cli:app`
+- `check_db` → `suhail_pipeline.utils.db_checker:app`

_bmad-output/project-context.md

@@ -9,12 +9,12 @@ sections_completed:
 
 # Project context for AI agents
 
-Lean rules for implementing code in **meshic-pipeline**. Prefer existing patterns in `src/meshic_pipeline/`. Full narrative lives in `docs/` and `docs/docs-distillate/`.
+Lean rules for implementing code in **suhail-pipeline**. Prefer existing patterns in `src/suhail_pipeline/`. Full narrative lives in `docs/` and `docs/docs-distillate/`.
 
 ## Technology stack and versions
 
 - **Language:** Python 3.9+ (`pyproject.toml`); **CI uses Python 3.11** — match CI when running tests locally.
-- **Package:** `meshic-pipeline` 0.1.0, `src` layout, entrypoints `meshic-pipeline` and `check_db`.
+- **Package:** `suhail-pipeline` 0.1.0, `src` layout, entrypoints `suhail-pipeline` and `check_db`.
 - **DB:** PostgreSQL + PostGIS; SQLAlchemy 2.x, GeoAlchemy2, Alembic; async via `asyncpg` where used.
 - **Geo / ETL:** GeoPandas, Shapely, h3/h3pandas, `mapbox-vector-tile`, mercantile, aiohttp.
 - **CLI / config:** Typer, Pydantic / pydantic-settings, YAML + `.env` for secrets.
@@ -31,7 +31,7 @@ Lean rules for implementing code in **meshic-pipeline**. Prefer existing pattern
 
 ### Code organization
 
-- **Package root:** `src/meshic_pipeline/` — new modules follow existing folders: `persistence/`, `enrichment/`, `geometry/`, `decoder/`, `downloader/`, `discovery/`.
+- **Package root:** `src/suhail_pipeline/` — new modules follow existing folders: `persistence/`, `enrichment/`, `geometry/`, `decoder/`, `downloader/`, `discovery/`.
 - **CLI:** New commands go through `cli.py` with Typer; keep help strings and safe defaults consistent with documented commands in `docs/CLI_COMMAND_AUDIT.md`.
 - **Migrations:** Schema changes require Alembic revisions under `alembic/versions/`; do not hand-edit production without a migration.
 

alembic/env.py

@@ -5,7 +5,7 @@
 from sqlalchemy import engine_from_config
 from sqlalchemy import pool
 from alembic import context
-from src.meshic_pipeline.persistence.models import Base
+from src.suhail_pipeline.persistence.models import Base
 
 # this is the Alembic Config object, which provides
 # access to the values within the .ini file in use.

alembic/versions/19c587b33197_add_critical_performance_indexes.py

@@ -1,6 +1,6 @@
 """add_critical_performance_indexes
 
-Critical database performance optimizations for the Meshic pipeline.
+Critical database performance optimizations for the Suhail pipeline.
 Based on docs/archive/legacy-root-md/DATABASE_ARCHITECTURE_ANALYSIS.md recommendations.
 
 Provides 60-80% performance improvement through safe index additions.