diff --git a/plan/CLAUDE.md b/plan/CLAUDE.md index 2890361..fcbccaa 100644 --- a/plan/CLAUDE.md +++ b/plan/CLAUDE.md @@ -34,5 +34,6 @@ A local Python pipeline that ingests a monthly BA15-Networking Excel workbook, c If a task's validation fails twice (one fix attempt after the first failure), stop working, do not improvise, and report: task number, what was tried, exact error text. Do not move to the next task and do not weaken the validation/acceptance criteria to force a pass. ## Known pending inputs (see plan/PREFERENCES.md) +- Debtors/DSO reporting is OUT OF SCOPE until a debtors aging data source exists — see PLAN.md section 9.2. Do not invent one from the workbook. - Board pack slide order/content — default proposed in PLAN.md section 7, may be revised after first real run. - Any additional branding preferences (fonts/colors already come from the template, but chart colors in matplotlib-rendered images need a decision — see PREFERENCES.md). diff --git a/plan/PLAN.md b/plan/PLAN.md index 64e8488..c634efc 100644 --- a/plan/PLAN.md +++ b/plan/PLAN.md @@ -136,7 +136,71 @@ Chart images (slides 4–5) are rendered by `dashboard.py`'s SVG chart functions 6. **Board Pack** — `boardpack.py` filling `templates/board_template.pptx` per section 7 table. 7. **CLI & packaging** — `cli.py` end-to-end orchestration; optionally package as a Claude skill via `skill-creator` for one-command monthly runs. -## 9. Risks / pending inputs +## 9. Exco enhancements — "path to full year" (branch `claude/exco-dashboard-enhancements`) + +### 9.1 Motivation +The v1 dashboard reports *position* (P&L vs budget, trend, concentration, pipeline, inventory, WIP). For an exco audience the missing layer is *trajectory*: whether the FY lands and what closes the gap. This iteration adds gap-to-budget bridging, pipeline coverage, order intake, margin quality, and an annuity/project split — and demotes low-materiality detail. + +### 9.2 Decisions (confirmed by user, 2026-07-05) +1. **Secured vs open pipeline split** (drives the bridge and coverage): + - `SECURED_STATUSES = {"Order Received", "Order Accept", "Order Take", "Invoiced", "Part-Invoiced", "Processed"}` — counted at full value (unweighted). + - `OPEN_STATUSES = {"Commit", "Forecast"}` — counted probability-weighted. + - Any status not in either set classifies as **open** (conservative — understates the secured position) and the dashboard shows a footnote listing unrecognized statuses when present. Do NOT fail loudly on new statuses; the source system adds them over time. +2. **Annuity vs project split**: keyword heuristic on `Description` (case-insensitive substring). `ANNUITY_KEYWORDS = ["support", "renewal", "annuity", "managed", "sla", "maintenance", "subscription", "licence", "license"]` — a module-level constant in `metrics.py` so it stays configurable. The dashboard must label the split "keyword-classified (approximate)". +3. **Order intake / bookings**: derived from month-over-month snapshot diffs (no bookings-date or bookings-budget source exists). Defined in 9.3. Renders only when ≥2 snapshots exist; otherwise show an "insufficient history" note. No bookings-vs-budget comparison in this iteration. +4. **Debtors / DSO**: out of scope — no debtors data in the workbook. Revisit when a debtors aging source exists. + +### 9.3 New metric definitions +- **Bridge components** (computed for Revenue and Gross Profit): + - `fy_budget` = `pnl.fy..bud`. + - `ytd_actual` = `pnl.ytd..act`. + - `secured_backlog` = Σ over Est rows with status ∈ SECURED_STATUSES of the raw value (unweighted). + - `weighted_open` = Σ over Est rows with status ∈ OPEN_STATUSES of value × probability weight (same weight rule as pipeline_weighted_totals). + - `remaining_gap` = `fy_budget − ytd_actual − secured_backlog − weighted_open` (positive = shortfall vs budget, negative = over-covered). + - Cross-check footnote: `ytd_actual + secured_backlog + weighted_open` vs the IP sheet's own `pnl.fy..est` — display the delta so a bottom-up/company-forecast divergence is visible. +- **Pipeline coverage** (Revenue and GP): `gap_to_cover = fy_budget − ytd_actual − secured_backlog`; + `coverage_weighted = weighted_open / gap_to_cover`; `coverage_gross = gross_open / gap_to_cover` (gross_open = unweighted Σ of OPEN_STATUSES rows). If `gap_to_cover <= 0`, coverage is `null` and the dashboard shows "budget fully covered by secured work". +- **Order intake** (per consecutive snapshot pair, computed in `history.py`): aggregate each snapshot's `pipeline.deals` + `actuals_deals` by FCID; an FCID is *secured* if ANY of its rows has a secured status. `intake(p2)` = Σ revenue (and GP, and count) of FCIDs secured in p2 but not secured in (or absent from) p1. First snapshot ⇒ `null`. +- **Negative-GP deal stats** (reporting month, from Act rows filtered to the reporting month as in `_month_actuals`): `negative_gp_count`, `negative_gp_value` (Σ of the negative GP amounts), plus prior-month count via history for the delta display. +- **GP margin trend**: per-period `gp / revenue` from `month_actuals` (fallback: P&L month GP / Sales Revenue), rendered as a %-scale line on the existing trend chart (right-hand axis). +- **Annuity split** (reporting-month Act rows, and FY-forward Est rows): `annuity_revenue`, `annuity_gp`, `project_revenue`, `project_gp`, `annuity_share = annuity_revenue / total` (null-safe). + +### 9.4 Snapshot schema additions (extends section 4; all keys optional for backward compat — renderers must tolerate their absence in older snapshots) +```json +{ + "pipeline": { + "by_status": {"secured": {"revenue": 0.0, "gp": 0.0}, "open_weighted": {"revenue": 0.0, "gp": 0.0}, "open_gross": {"revenue": 0.0, "gp": 0.0}, "unrecognized_statuses": []} + }, + "bridge": { + "revenue": {"fy_budget": 0.0, "ytd_actual": 0.0, "secured_backlog": 0.0, "weighted_open": 0.0, "remaining_gap": 0.0, "fy_est_source": 0.0}, + "gp": {"...": "same shape"} + }, + "coverage": {"revenue": {"gap_to_cover": 0.0, "weighted": 0.0, "gross": 0.0}, "gp": {"...": "same shape"}}, + "deal_quality": {"negative_gp_count": 0, "negative_gp_value": 0.0}, + "annuity": {"month_act": {"annuity_revenue": 0.0, "annuity_gp": 0.0, "project_revenue": 0.0, "project_gp": 0.0, "annuity_share": 0.0}, "fy_est": {"...": "same shape"}} +} +``` +Order intake is NOT stored in snapshots (it is derived across snapshots in `history.py` at render time). + +### 9.5 Dashboard layout (15-minute exco flow — reorder to this sequence) +1. **Key Metrics** — existing tiles + new: "Deals at negative GP (month): X (prior: Y)" and "Annuity share of revenue: Z%". +2. **Actual vs Budget** — existing chart + table. +3. **Path to Full Year** (NEW) — SVG waterfall: FY Budget → YTD Actual → Secured backlog → Weighted pipeline → Remaining gap; metric toggle Revenue/GP; coverage KPI tiles ("Open pipeline covers gap X.Xx weighted / Y.Yx gross"); cross-check footnote per 9.3. +4. **Order Intake** (NEW) — bar per period of derived intake (revenue), count annotation; "insufficient history" note when <2 snapshots. +5. **Revenue & GP Trend** — existing chart + GP-margin % line on a right-hand % axis (distinct color `#262579`, dashed). +6. **GP — Customers & Pipeline** — existing donut/bars + annuity vs project stacked bar (Act month + FY Est). +7. **Risk tiles** — WIP section unchanged EXCEPT remove the duplicate `_wip_overdue_bars` card (keep the table); Inventory demoted to a single tile: "Inventory on hand: R X.XM — % aged >180d" (drop the two full aging cards; keep detail available in the boardpack appendix only). + +Waterfall colors: budget `#CCCCCC`, YTD actual NEC Blue `#1159A5`, secured NEC Green `#C1D552`, weighted open Trust Blue `#262579`, remaining gap Brighter Orange `#E37739`. All other brand rules per PREFERENCES.md. + +### 9.6 Board pack additions +- Executive summary slide: add bullets for remaining gap, coverage ratio, negative-GP count, annuity share (max 6 bullets total — drop the least material existing bullet if needed). +- New slide after P&L summary: "Path to Full Year" — matplotlib waterfall PNG (same figures/colors as 9.5.3), layout index 3 (`Content Slide 1`). + +### 9.7 Environment prerequisite +The dev machine now runs Python 3.14; `pandas==2.2.3` and `matplotlib==3.9.2` have no cp314 wheels and fail to install. Before any code task: bump `requirements.txt` pins to the current latest versions that ship Python 3.14 wheels (verify with `pip install -r requirements.txt` succeeding), then run the full test suite and fix any deprecation breakage BEFORE starting feature work. + +## 10. Risks / pending inputs - Branding is fully specified: all chart/dashboard colors, fonts, and slide conventions are DECIDED in `plan/PREFERENCES.md` (NEC XON brand guidelines). Tasks 8 and 9 MUST use those exact hex values and font stacks. - Prior-year monthly files: assumed same layout as `6BC_IP.xlsx`; if a prior file's schema differs, ingestion must fail loudly (STOP), not silently coerce. - `WIP` sheet has no fixed header row — ingestion must scan for it; if not found, STOP rather than guess. diff --git a/plan/TASKS.md b/plan/TASKS.md index 6bf5796..c4b7c5f 100644 --- a/plan/TASKS.md +++ b/plan/TASKS.md @@ -348,3 +348,160 @@ Use `skill-creator` to scaffold a project skill `run-monthly-report` that wraps ## Final step for every task After each task's validation passes, `git add` the new/changed files (never `input/`, `data/`, `output/`) and commit with a message describing that task. Do not push until Task 11 passes. + +--- +--- + +# Phase 2 — Exco enhancements (branch `claude/exco-dashboard-enhancements`) +Read PLAN.md section 9 before starting. The executor preamble at the top of this file applies to every task below. Definitions (SECURED_STATUSES, OPEN_STATUSES, ANNUITY_KEYWORDS, bridge/coverage/intake formulas) live in PLAN.md 9.2–9.3 — implement them exactly; do not re-derive. + +### Task 13 — Environment: fix dependency pins for Python 3.14 +`requirements.txt` pins (`pandas==2.2.3`, `matplotlib==3.9.2`) have no Python 3.14 wheels. Bump each pinned package to the current latest release that installs cleanly on the host Python (keep exact `==` pins). Fix any deprecation/API breakage the bumps cause in existing code. + +**Validation command:** +``` +pip install -r requirements.txt && python -m pytest tests/ -q +``` +**Expected result:** install succeeds (no source builds attempted/failed), full suite passes, exit code 0. + +--- + +### Task 14 — Extend synthetic fixture for phase-2 data shapes +**File:** `tests/fixtures/make_synthetic_workbook.py` + +Extend (keep all existing rows/shapes so phase-1 tests still pass): +- `Est` rows: add rows covering EVERY status in SECURED_STATUSES ∪ OPEN_STATUSES, plus one row with an unrecognized status `"Weird New Status"`. Give Commit/Forecast rows probabilities < 100 so weighting is exercised. +- `Est`/`Act` Descriptions: add rows whose Description contains annuity keywords (e.g. "Synthetic Support Renewal 1", "Synthetic Managed Service 2") and rows without any keyword. +- `Act` rows: add ≥2 rows with negative GP in the reporting month. +- Keep every figure synthetic and round. + +**Validation command:** +``` +python tests/fixtures/make_synthetic_workbook.py && python -m pytest tests/ -q +``` +**Expected result:** fixture regenerates; existing suite still passes. + +--- + +### Task 15 — Metrics: status split, bridge, coverage, deal quality, annuity +**File:** `src/fdat/metrics.py` (+ tests in `tests/test_metrics.py`) + +Add module constants `SECURED_STATUSES`, `OPEN_STATUSES`, `ANNUITY_KEYWORDS` (values per PLAN.md 9.2) and pure functions: +```python +def classify_status(status) -> str: ... # 'secured' | 'open' ('open' for unrecognized/None) +def pipeline_by_status(est_df, probability_scale) -> dict: ... # PLAN.md 9.4 pipeline.by_status shape, incl. unrecognized_statuses list +def bridge_components(fy_budget, ytd_actual, secured_backlog, weighted_open, fy_est_source) -> dict: ... # 9.4 bridge. shape +def coverage_ratios(fy_budget, ytd_actual, secured_backlog, weighted_open, gross_open) -> dict: ... # 9.4 coverage. shape; None when gap_to_cover <= 0 +def negative_gp_stats(month_act_df) -> dict: ... # {'negative_gp_count': int, 'negative_gp_value': float} +def is_annuity(description) -> bool: ... # keyword substring match, case-insensitive, None-safe +def annuity_split(deals_df, probability_scale=None) -> dict: ... # 9.4 annuity. shape; weighted when probability_scale given +``` +Unit tests: ≥2 cases per function including edge cases (empty df, zero budget/gap, None description, unrecognized status lands in 'open' + is reported). + +**Validation command:** +``` +python -m pytest tests/test_metrics.py -q +``` +**Expected result:** all pass. + +--- + +### Task 16 — Snapshot: persist the new metric blocks +**File:** `src/fdat/snapshot.py` (+ tests in `tests/test_snapshot.py`) + +In `build_snapshot`, populate the PLAN.md 9.4 additions (`pipeline.by_status`, `bridge`, `coverage`, `deal_quality`, `annuity`) using Task 15 functions. Bridge/coverage computed for both "Sales Revenue" and "Gross Profit" P&L lines. `deal_quality` uses the same reporting-month Act filter as `_month_actuals`. `annuity.month_act` from reporting-month Act rows; `annuity.fy_est` from all Est rows (weighted). + +**Validation command:** +``` +python -m pytest tests/test_snapshot.py -q +``` +**Expected result:** all pass, including new assertions that each new block exists with the 9.4 shape and internally consistent values (e.g. remaining_gap arithmetic). + +--- + +### Task 17 — History: order intake + GP-margin series +**File:** `src/fdat/history.py` (+ tests in `tests/test_history.py`) + +Implement per PLAN.md 9.3: +```python +def compute_order_intake(snapshots: list) -> list: ... +# [{'period': 'YYYY-MM', 'intake_revenue': float|None, 'intake_gp': float|None, 'deal_count': int|None}] +# First snapshot -> None values. FCID aggregation + ANY-row-secured rule per 9.3. +def compute_gp_margin_series(snapshots: list) -> list: ... +# [{'period': 'YYYY-MM', 'margin': float|None}] from month_actuals with P&L fallback per 9.3. +``` +Snapshots missing the new blocks (older history) must not crash either function. + +**Validation command:** +``` +python -m pytest tests/test_history.py -q +``` +**Expected result:** all pass, including a two-snapshot intake case where a deal transitions Forecast→Order Received and a brand-new secured FCID appears (both counted), and an unchanged secured deal (not counted). + +--- + +### Task 18 — Dashboard: "Path to Full Year" waterfall + coverage tiles +**Read skill: `dataviz` first.** **File:** `src/fdat/dashboard.py` (+ tests in `tests/test_dashboard.py`) + +New section per PLAN.md 9.5 item 3, placed directly after the Actual-vs-Budget cards: hand-rolled SVG waterfall (FY Budget → YTD Actual → Secured backlog → Weighted pipeline → Remaining gap) with Revenue/GP toggle (same client-side pattern as the AVB chart), coverage KPI tiles, unrecognized-status footnote, and the fy_est cross-check footnote. Colors per 9.5. Sections must degrade gracefully (render an explanatory note) when the snapshot lacks the `bridge`/`coverage` blocks. + +**Validation command:** +``` +python -m pytest tests/test_dashboard.py -q +``` +**Expected result:** all pass, including: output contains the waterfall section for a phase-2 snapshot; output contains a graceful note (not a crash) for a legacy snapshot dict without the new blocks; still no `http://`/`https://` in the HTML. + +--- + +### Task 19 — Dashboard: GP-margin trend line, negative-GP KPI, annuity split +**File:** `src/fdat/dashboard.py` (+ tests) + +Per PLAN.md 9.5 items 1, 5, 6: GP-margin % line on the existing trend chart with a right-hand % axis (color `#262579`, dashed, own toggle checkbox); KPI tile "Deals at negative GP (month)" showing current count and prior-month count (prior from history — pass it into `render_dashboard` via the existing series/budget_series pattern, not a new global); annuity vs project stacked bar (month Act + FY Est) labeled "keyword-classified (approximate)". + +**Validation command:** +``` +python -m pytest tests/test_dashboard.py -q +``` +**Expected result:** all pass. + +--- + +### Task 20 — Dashboard: order-intake section + exco reordering + demotions +**File:** `src/fdat/dashboard.py` (+ tests) + +- Order-intake bars (revenue per period, deal-count annotation); when <2 snapshots render the "insufficient history" note (PLAN.md 9.2.3). +- Reorder sections to the 9.5 sequence. +- Demote inventory to the single KPI tile defined in 9.5.7 (remove the two aging cards from the dashboard). +- Remove the duplicate `_wip_overdue_bars` card (keep the table and WIP KPI tiles). + +**Validation command:** +``` +python -m pytest tests/test_dashboard.py -q && python -c " +from tests.fixtures.make_synthetic_workbook import build_workbook +import tempfile, os +# smoke: full render on synthetic data, then manual browser check +" +``` +**Expected result:** tests pass. Then run the full CLI against the synthetic fixture, open the dashboard in a browser, and confirm: new section order, waterfall renders, no console errors. Note the manual check outcome in the commit message. + +--- + +### Task 21 — Board pack: exec-summary bullets + bridge slide +**Read skill: `pptx` first.** **File:** `src/fdat/boardpack.py` (+ tests in `tests/test_boardpack.py`) + +Per PLAN.md 9.6: extend exec-summary bullets (cap 6) and insert the "Path to Full Year" slide (matplotlib waterfall PNG, brand colors, layout index 3 `Content Slide 1`) after the P&L summary slide. Keep the fail-loudly layout-name check. Move the detailed inventory aging table to the appendix slide. + +**Validation command:** +``` +python -m pytest tests/test_boardpack.py -q +``` +**Expected result:** all pass; slide count assertion updated for the added slide. + +--- + +### Task 22 — Full suite + end-to-end run +**Validation command:** +``` +python -m pytest tests/ -q && python -m src.fdat.cli run tests/fixtures/synthetic_sample.xlsx --period 2024-01 --history-dir /tmp/fdat_p2_hist --template templates/board_template.pptx --output-dir /tmp/fdat_p2_out +``` +**Expected result:** suite green; CLI writes dashboard + boardpack with no traceback. STOP rule applies.