diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml index b31de8f..6c97b3e 100644 --- a/.github/workflows/e2e.yml +++ b/.github/workflows/e2e.yml @@ -107,9 +107,11 @@ jobs: # ── Seed demo users ────────────────────────────────────────────── # E2E auth fixtures expect demo users (matt@bandrcapital.com, demo@bandrcapital.com). # Uses the existing seed module at app.database.seed which creates users, properties, and deals. + # Fail the job if seeding fails — without demo users every auth-gated test + # downstream fails and the run wastes 30+ minutes producing meaningless red. - name: Seed database with demo data working-directory: backend - run: python -m app.database.seed || echo "::warning::Database seeding failed — auth E2E tests may fail" + run: python -m app.database.seed # ── Start backend ──────────────────────────────────────────────── - name: Start backend server diff --git a/backend/requirements-ci.txt b/backend/requirements-ci.txt index 41cf242..633bebd 100644 --- a/backend/requirements-ci.txt +++ b/backend/requirements-ci.txt @@ -40,10 +40,18 @@ structlog>=23.1.0 aiohttp>=3.8.0 apscheduler>=3.10.0 +# Reporting / Charts +# Required by app.services.report_templates and report_charts. Without these, +# tests/test_services/test_report_templates.py fails at import time and pytest +# exits non-zero even though the actual tests would have skipped cleanly. +reportlab>=4.0.0,<5.0.0 +matplotlib>=3.8.0,<4.0.0 + # Authentication & Security PyJWT[crypto]>=2.8.0,<3.0.0 passlib[bcrypt]>=1.7.4 -bcrypt>=4.1.0,<5.0.0 +# bcrypt pinned to <4 — see backend/requirements.txt for context. +bcrypt<4.0 httpx>=0.26.0 # Utilities diff --git a/backend/requirements.txt b/backend/requirements.txt index c140609..4be2e32 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -47,7 +47,11 @@ apscheduler>=3.10.0,<4.0.0 # Authentication & Security PyJWT[crypto]>=2.12.0,<3.0.0 passlib[bcrypt]>=1.7.4,<2.0.0 -bcrypt>=4.1.0,<5.0.0 +# bcrypt pinned to <4 because passlib 1.7.4 reads bcrypt.__about__ which was +# removed in bcrypt 4.0 (AttributeError: module 'bcrypt' has no attribute '__about__'). +# passlib has no 1.7.5 release, so the dependency stays pinned until either passlib +# ships a fix or this codebase migrates off passlib. +bcrypt<4.0 httpx>=0.26.0,<1.0.0 # Utilities diff --git a/docs/zod-parity-followups.md b/docs/zod-parity-followups.md index 73f0298..1d8b3a5 100644 --- a/docs/zod-parity-followups.md +++ b/docs/zod-parity-followups.md @@ -2,49 +2,33 @@ The `check-zod-parity.sh` PostToolUse hook compares fields declared in `backend/app/schemas/*.py` to fields in the matching `src/lib/api/schemas/.ts`. Anything in the Pydantic schema but missing from Zod is silently dropped at parse time — Zod's default `z.object()` strips unknowns. -Survey run on 2026-04-29 against all 22 backend schema files identified four files with real drift. `deal.ts` was partially fixed (12 `DealResponse` fields added); the rest are tracked here. +Survey run on 2026-04-29 against all 22 backend schema files identified four files with real drift. **All four have since been closed** — live parity check returns zero missing fields across `property.ts`, `reporting.ts`, `construction.ts`, and `deal.ts` as of 2026-04-29. -## Priority 1 — `property.ts` +## Status -37 missing fields. `property.py` is the largest schema and most likely to be silently dropping data the UI needs. +| Priority | File | Status | Closing commit | +|----------|------|--------|----------------| +| 1 | `property.ts` | ✅ Closed | `c65eaf9` — fix(schemas): add flat backendPropertySchema for PropertyResponse parity (#4) | +| 2 | `reporting.ts` | ✅ Closed | `bf78179` — fix(schemas): add Zod schemas for all reporting response classes (#5) | +| 3 | `construction.ts` | ✅ Closed | `e1df8f9` — fix(schemas): close construction.ts Zod parity gap (#3) | +| 4 | `deal.ts` sub-class artifacts | ✅ Closed | `a184058` — fix(schemas): close deal.ts sub-class artifact parity gaps (#6) | -Missing (first 10): `acquisition_date, address, avg_rent_per_sf, avg_rent_per_unit, cap_rate, city, county, current_value, data_source, description` — plus 27 more. +To re-verify locally: -To do: -1. Read both `backend/app/schemas/property.py` and `src/lib/api/schemas/property.ts`. -2. Identify which Pydantic class each missing field belongs to (likely `PropertyResponse` vs `PropertyCreate`/`PropertyUpdate`/sub-classes). -3. Add the response-only fields to the Zod input shape with `.nullable().optional()` per CLAUDE.md convention. -4. Run `npm run test:run -- src/features/property` to confirm no regressions. +```bash +for stem in property reporting construction deal; do + py_fields=$(grep -E '^[[:space:]]{4}[a-z][a-z0-9_]*:[[:space:]]' backend/app/schemas/${stem}.py \ + | sed -E 's/^[[:space:]]+([a-z][a-z0-9_]*):.*/\1/' \ + | grep -vE '^(model_config|class_config|_)' | sort -u) + zod_fields=$(grep -E '^[[:space:]]+[a-z][a-z0-9_]*:[[:space:]]*z\.' src/lib/api/schemas/${stem}.ts \ + | sed -E 's/^[[:space:]]+([a-z][a-z0-9_]*):.*/\1/' | sort -u) + echo "=== $stem ===" + comm -23 <(printf '%s\n' "$py_fields") <(printf '%s\n' "$zod_fields") +done +``` -## Priority 2 — `reporting.ts` +## Hook limitation (open) -43+ missing fields. Likely a mix of report config schemas, scheduled report fields, and template metadata. +`check-zod-parity.sh` is regex-based: it grabs every indented `name: type` line in the `.py` file and checks against every Zod field in the `.ts` file, with no awareness of Pydantic class boundaries. That's why a single-file Pydantic module with multiple response classes can generate noise even when each class has its own (correctly-shaped) Zod counterpart — the hook concatenates fields across classes. -Missing (first 10): `category, completed_at, config, configurable, created_at, created_by, day_of_month, day_of_week, default_height, default_width` — plus 33 more. - -Approach: same as property.ts. Reporting code is newer (recently shipped — see commit `6cbc6a3 feat(reports)`) so the gap may reflect intentional separation between backend admin schemas and frontend wire schemas. Verify before adding fields wholesale. - -## Priority 3 — `construction.ts` - -5 missing fields: `count, results, rows_imported, rows_updated, total_value`. - -Small and tractable. These look like batch-import response fields — confirm whether the frontend triggers construction imports and needs the result stats. - -## Priority 4 — `deal.ts` sub-class artifacts - -23 fields the hook flags on `deal.py` are NOT `DealResponse` fields — they belong to sibling response classes that have no Zod schema at all: - -- `ProformaFieldValue`, `ProformaFieldGroup`, `ProformaReturnsResponse` — `category`, `field_name`, `fields`, `groups`, `value_numeric`, `value_text` -- `WatchlistStatusResponse` — `is_watched` -- `StageChangeLogResponse`, `StageHistoryResponse` — `changed_by_user_id`, `history`, `new_stage`, `old_stage`, `reason` -- `KanbanBoardResponse` — `stage_counts`, `stages`, `total_deals` -- `StageMappingResponse` — `folder_to_stage` -- `DealCursorPaginatedResponse` — `has_more`, `next_cursor`, `prev_cursor` -- `DealListResponse` — `items`, `page`, `page_size` (already covered partially via `dealsListResponseSchema`) -- Shared — `deal_id`, `deal_name`, `total` - -If the frontend calls the corresponding endpoints (`/deals/{id}/proforma-returns`, `/deals/{id}/watchlist`, `/deals/{id}/stage-history`, `/deals/kanban`, etc.), each needs its own Zod schema. If those endpoints are unused, the noise can be silenced by extending `check-zod-parity.sh` to scope to a single Pydantic class. - -## Hook limitation - -`check-zod-parity.sh` is regex-based: it grabs every indented `name: type` line in the `.py` file and checks against every Zod field in the `.ts` file, with no awareness of Pydantic class boundaries. That's why a single-file Pydantic module with 8 response classes generates noise like the deal.py case above. A future iteration could parse the AST (via Python `ast` module) and emit one diff per class — out of scope for now, but logged here. +A future iteration could parse the AST (via Python `ast` module) and emit one diff per class. Out of scope for the parity-closure work, but worth logging here so the next person hitting hook noise has context. diff --git a/playwright.config.ts b/playwright.config.ts index 6027e19..2d47e6c 100644 --- a/playwright.config.ts +++ b/playwright.config.ts @@ -20,7 +20,9 @@ export default defineConfig({ forbidOnly: isCI, /* Default retries/workers — overridden per-project */ retries: 0, - workers: isCI ? 1 : undefined, + /* CI: 4 workers on a 4-core ubuntu-latest runner. Was 1 — caused 403 tests to + * exhaust the 60m job timeout on every run. Drop to 2 if memory pressure surfaces. */ + workers: isCI ? 4 : undefined, reporter: isCI ? [['json', { outputFile: 'playwright-report/results.json' }], ['html', { open: 'never' }]] : 'html',