Plan: exco 'path to full year' dashboard enhancements

plan/CLAUDE.md

@@ -34,5 +34,6 @@ A local Python pipeline that ingests a monthly BA15-Networking Excel workbook, c
 If a task's validation fails twice (one fix attempt after the first failure), stop working, do not improvise, and report: task number, what was tried, exact error text. Do not move to the next task and do not weaken the validation/acceptance criteria to force a pass.
 
 ## Known pending inputs (see plan/PREFERENCES.md)
+- Debtors/DSO reporting is OUT OF SCOPE until a debtors aging data source exists — see PLAN.md section 9.2. Do not invent one from the workbook.
 - Board pack slide order/content — default proposed in PLAN.md section 7, may be revised after first real run.
 - Any additional branding preferences (fonts/colors already come from the template, but chart colors in matplotlib-rendered images need a decision — see PREFERENCES.md).

plan/PLAN.md

@@ -136,7 +136,71 @@ Chart images (slides 4–5) are rendered by `dashboard.py`'s SVG chart functions
 6. **Board Pack** — `boardpack.py` filling `templates/board_template.pptx` per section 7 table.
 7. **CLI & packaging** — `cli.py` end-to-end orchestration; optionally package as a Claude skill via `skill-creator` for one-command monthly runs.
 
-## 9. Risks / pending inputs
+## 9. Exco enhancements — "path to full year" (branch `claude/exco-dashboard-enhancements`)
+
+### 9.1 Motivation
+The v1 dashboard reports *position* (P&L vs budget, trend, concentration, pipeline, inventory, WIP). For an exco audience the missing layer is *trajectory*: whether the FY lands and what closes the gap. This iteration adds gap-to-budget bridging, pipeline coverage, order intake, margin quality, and an annuity/project split — and demotes low-materiality detail.
+
+### 9.2 Decisions (confirmed by user, 2026-07-05)
+1. **Secured vs open pipeline split** (drives the bridge and coverage):
+   - `SECURED_STATUSES = {"Order Received", "Order Accept", "Order Take", "Invoiced", "Part-Invoiced", "Processed"}` — counted at full value (unweighted).
+   - `OPEN_STATUSES = {"Commit", "Forecast"}` — counted probability-weighted.
+   - Any status not in either set classifies as **open** (conservative — understates the secured position) and the dashboard shows a footnote listing unrecognized statuses when present. Do NOT fail loudly on new statuses; the source system adds them over time.
+2. **Annuity vs project split**: keyword heuristic on `Description` (case-insensitive substring). `ANNUITY_KEYWORDS = ["support", "renewal", "annuity", "managed", "sla", "maintenance", "subscription", "licence", "license"]` — a module-level constant in `metrics.py` so it stays configurable. The dashboard must label the split "keyword-classified (approximate)".
+3. **Order intake / bookings**: derived from month-over-month snapshot diffs (no bookings-date or bookings-budget source exists). Defined in 9.3. Renders only when ≥2 snapshots exist; otherwise show an "insufficient history" note. No bookings-vs-budget comparison in this iteration.
+4. **Debtors / DSO**: out of scope — no debtors data in the workbook. Revisit when a debtors aging source exists.
+
+### 9.3 New metric definitions
+- **Bridge components** (computed for Revenue and Gross Profit):
+  - `fy_budget` = `pnl.fy.<item>.bud`.
+  - `ytd_actual` = `pnl.ytd.<item>.act`.
+  - `secured_backlog` = Σ over Est rows with status ∈ SECURED_STATUSES of the raw value (unweighted).
+  - `weighted_open` = Σ over Est rows with status ∈ OPEN_STATUSES of value × probability weight (same weight rule as pipeline_weighted_totals).
+  - `remaining_gap` = `fy_budget − ytd_actual − secured_backlog − weighted_open` (positive = shortfall vs budget, negative = over-covered).
+  - Cross-check footnote: `ytd_actual + secured_backlog + weighted_open` vs the IP sheet's own `pnl.fy.<item>.est` — display the delta so a bottom-up/company-forecast divergence is visible.
+- **Pipeline coverage** (Revenue and GP): `gap_to_cover = fy_budget − ytd_actual − secured_backlog`;
+  `coverage_weighted = weighted_open / gap_to_cover`; `coverage_gross = gross_open / gap_to_cover` (gross_open = unweighted Σ of OPEN_STATUSES rows). If `gap_to_cover <= 0`, coverage is `null` and the dashboard shows "budget fully covered by secured work".
+- **Order intake** (per consecutive snapshot pair, computed in `history.py`): aggregate each snapshot's `pipeline.deals` + `actuals_deals` by FCID; an FCID is *secured* if ANY of its rows has a secured status. `intake(p2)` = Σ revenue (and GP, and count) of FCIDs secured in p2 but not secured in (or absent from) p1. First snapshot ⇒ `null`.
+- **Negative-GP deal stats** (reporting month, from Act rows filtered to the reporting month as in `_month_actuals`): `negative_gp_count`, `negative_gp_value` (Σ of the negative GP amounts), plus prior-month count via history for the delta display.
+- **GP margin trend**: per-period `gp / revenue` from `month_actuals` (fallback: P&L month GP / Sales Revenue), rendered as a %-scale line on the existing trend chart (right-hand axis).
+- **Annuity split** (reporting-month Act rows, and FY-forward Est rows): `annuity_revenue`, `annuity_gp`, `project_revenue`, `project_gp`, `annuity_share = annuity_revenue / total` (null-safe).
+
+### 9.4 Snapshot schema additions (extends section 4; all keys optional for backward compat — renderers must tolerate their absence in older snapshots)
+```json
+{
+  "pipeline": {
+    "by_status": {"secured": {"revenue": 0.0, "gp": 0.0}, "open_weighted": {"revenue": 0.0, "gp": 0.0}, "open_gross": {"revenue": 0.0, "gp": 0.0}, "unrecognized_statuses": []}
+  },
+  "bridge": {
+    "revenue": {"fy_budget": 0.0, "ytd_actual": 0.0, "secured_backlog": 0.0, "weighted_open": 0.0, "remaining_gap": 0.0, "fy_est_source": 0.0},
+    "gp": {"...": "same shape"}
+  },
+  "coverage": {"revenue": {"gap_to_cover": 0.0, "weighted": 0.0, "gross": 0.0}, "gp": {"...": "same shape"}},
+  "deal_quality": {"negative_gp_count": 0, "negative_gp_value": 0.0},
+  "annuity": {"month_act": {"annuity_revenue": 0.0, "annuity_gp": 0.0, "project_revenue": 0.0, "project_gp": 0.0, "annuity_share": 0.0}, "fy_est": {"...": "same shape"}}
+}
+```
+Order intake is NOT stored in snapshots (it is derived across snapshots in `history.py` at render time).
+
+### 9.5 Dashboard layout (15-minute exco flow — reorder to this sequence)
+1. **Key Metrics** — existing tiles + new: "Deals at negative GP (month): X (prior: Y)" and "Annuity share of revenue: Z%".
+2. **Actual vs Budget** — existing chart + table.
+3. **Path to Full Year** (NEW) — SVG waterfall: FY Budget → YTD Actual → Secured backlog → Weighted pipeline → Remaining gap; metric toggle Revenue/GP; coverage KPI tiles ("Open pipeline covers gap X.Xx weighted / Y.Yx gross"); cross-check footnote per 9.3.
+4. **Order Intake** (NEW) — bar per period of derived intake (revenue), count annotation; "insufficient history" note when <2 snapshots.
+5. **Revenue & GP Trend** — existing chart + GP-margin % line on a right-hand % axis (distinct color `#262579`, dashed).
+6. **GP — Customers & Pipeline** — existing donut/bars + annuity vs project stacked bar (Act month + FY Est).
+7. **Risk tiles** — WIP section unchanged EXCEPT remove the duplicate `_wip_overdue_bars` card (keep the table); Inventory demoted to a single tile: "Inventory on hand: R X.XM — % aged >180d" (drop the two full aging cards; keep detail available in the boardpack appendix only).
+
+Waterfall colors: budget `#CCCCCC`, YTD actual NEC Blue `#1159A5`, secured NEC Green `#C1D552`, weighted open Trust Blue `#262579`, remaining gap Brighter Orange `#E37739`. All other brand rules per PREFERENCES.md.
+
+### 9.6 Board pack additions
+- Executive summary slide: add bullets for remaining gap, coverage ratio, negative-GP count, annuity share (max 6 bullets total — drop the least material existing bullet if needed).
+- New slide after P&L summary: "Path to Full Year" — matplotlib waterfall PNG (same figures/colors as 9.5.3), layout index 3 (`Content Slide 1`).
+
+### 9.7 Environment prerequisite
+The dev machine now runs Python 3.14; `pandas==2.2.3` and `matplotlib==3.9.2` have no cp314 wheels and fail to install. Before any code task: bump `requirements.txt` pins to the current latest versions that ship Python 3.14 wheels (verify with `pip install -r requirements.txt` succeeding), then run the full test suite and fix any deprecation breakage BEFORE starting feature work.
+
+## 10. Risks / pending inputs
 - Branding is fully specified: all chart/dashboard colors, fonts, and slide conventions are DECIDED in `plan/PREFERENCES.md` (NEC XON brand guidelines). Tasks 8 and 9 MUST use those exact hex values and font stacks.
 - Prior-year monthly files: assumed same layout as `6BC_IP.xlsx`; if a prior file's schema differs, ingestion must fail loudly (STOP), not silently coerce.
 - `WIP` sheet has no fixed header row — ingestion must scan for it; if not found, STOP rather than guess.

plan/TASKS.md

@@ -348,3 +348,160 @@ Use `skill-creator` to scaffold a project skill `run-monthly-report` that wraps
 
 ## Final step for every task
 After each task's validation passes, `git add` the new/changed files (never `input/`, `data/`, `output/`) and commit with a message describing that task. Do not push until Task 11 passes.
+
+---
+---
+
+# Phase 2 — Exco enhancements (branch `claude/exco-dashboard-enhancements`)
+Read PLAN.md section 9 before starting. The executor preamble at the top of this file applies to every task below. Definitions (SECURED_STATUSES, OPEN_STATUSES, ANNUITY_KEYWORDS, bridge/coverage/intake formulas) live in PLAN.md 9.2–9.3 — implement them exactly; do not re-derive.
+
+### Task 13 — Environment: fix dependency pins for Python 3.14
+`requirements.txt` pins (`pandas==2.2.3`, `matplotlib==3.9.2`) have no Python 3.14 wheels. Bump each pinned package to the current latest release that installs cleanly on the host Python (keep exact `==` pins). Fix any deprecation/API breakage the bumps cause in existing code.
+
+**Validation command:**
+```
+pip install -r requirements.txt && python -m pytest tests/ -q
+```
+**Expected result:** install succeeds (no source builds attempted/failed), full suite passes, exit code 0.
+
+---
+
+### Task 14 — Extend synthetic fixture for phase-2 data shapes
+**File:** `tests/fixtures/make_synthetic_workbook.py`
+
+Extend (keep all existing rows/shapes so phase-1 tests still pass):
+- `Est` rows: add rows covering EVERY status in SECURED_STATUSES ∪ OPEN_STATUSES, plus one row with an unrecognized status `"Weird New Status"`. Give Commit/Forecast rows probabilities < 100 so weighting is exercised.
+- `Est`/`Act` Descriptions: add rows whose Description contains annuity keywords (e.g. "Synthetic Support Renewal 1", "Synthetic Managed Service 2") and rows without any keyword.
+- `Act` rows: add ≥2 rows with negative GP in the reporting month.
+- Keep every figure synthetic and round.
+
+**Validation command:**
+```
+python tests/fixtures/make_synthetic_workbook.py && python -m pytest tests/ -q
+```
+**Expected result:** fixture regenerates; existing suite still passes.
+
+---
+
+### Task 15 — Metrics: status split, bridge, coverage, deal quality, annuity
+**File:** `src/fdat/metrics.py` (+ tests in `tests/test_metrics.py`)
+
+Add module constants `SECURED_STATUSES`, `OPEN_STATUSES`, `ANNUITY_KEYWORDS` (values per PLAN.md 9.2) and pure functions:
+```python
+def classify_status(status) -> str: ...          # 'secured' | 'open' ('open' for unrecognized/None)
+def pipeline_by_status(est_df, probability_scale) -> dict: ...  # PLAN.md 9.4 pipeline.by_status shape, incl. unrecognized_statuses list
+def bridge_components(fy_budget, ytd_actual, secured_backlog, weighted_open, fy_est_source) -> dict: ...  # 9.4 bridge.<metric> shape
+def coverage_ratios(fy_budget, ytd_actual, secured_backlog, weighted_open, gross_open) -> dict: ...      # 9.4 coverage.<metric> shape; None when gap_to_cover <= 0
+def negative_gp_stats(month_act_df) -> dict: ...  # {'negative_gp_count': int, 'negative_gp_value': float}
+def is_annuity(description) -> bool: ...          # keyword substring match, case-insensitive, None-safe
+def annuity_split(deals_df, probability_scale=None) -> dict: ... # 9.4 annuity.<block> shape; weighted when probability_scale given
+```
+Unit tests: ≥2 cases per function including edge cases (empty df, zero budget/gap, None description, unrecognized status lands in 'open' + is reported).
+
+**Validation command:**
+```
+python -m pytest tests/test_metrics.py -q
+```
+**Expected result:** all pass.
+
+---
+
+### Task 16 — Snapshot: persist the new metric blocks
+**File:** `src/fdat/snapshot.py` (+ tests in `tests/test_snapshot.py`)
+
+In `build_snapshot`, populate the PLAN.md 9.4 additions (`pipeline.by_status`, `bridge`, `coverage`, `deal_quality`, `annuity`) using Task 15 functions. Bridge/coverage computed for both "Sales Revenue" and "Gross Profit" P&L lines. `deal_quality` uses the same reporting-month Act filter as `_month_actuals`. `annuity.month_act` from reporting-month Act rows; `annuity.fy_est` from all Est rows (weighted).
+
+**Validation command:**
+```
+python -m pytest tests/test_snapshot.py -q
+```
+**Expected result:** all pass, including new assertions that each new block exists with the 9.4 shape and internally consistent values (e.g. remaining_gap arithmetic).
+
+---
+
+### Task 17 — History: order intake + GP-margin series
+**File:** `src/fdat/history.py` (+ tests in `tests/test_history.py`)
+
+Implement per PLAN.md 9.3:
+```python
+def compute_order_intake(snapshots: list) -> list: ...
+# [{'period': 'YYYY-MM', 'intake_revenue': float|None, 'intake_gp': float|None, 'deal_count': int|None}]
+# First snapshot -> None values. FCID aggregation + ANY-row-secured rule per 9.3.
+def compute_gp_margin_series(snapshots: list) -> list: ...
+# [{'period': 'YYYY-MM', 'margin': float|None}] from month_actuals with P&L fallback per 9.3.
+```
+Snapshots missing the new blocks (older history) must not crash either function.
+
+**Validation command:**
+```
+python -m pytest tests/test_history.py -q
+```
+**Expected result:** all pass, including a two-snapshot intake case where a deal transitions Forecast→Order Received and a brand-new secured FCID appears (both counted), and an unchanged secured deal (not counted).
+
+---
+
+### Task 18 — Dashboard: "Path to Full Year" waterfall + coverage tiles
+**Read skill: `dataviz` first.** **File:** `src/fdat/dashboard.py` (+ tests in `tests/test_dashboard.py`)
+
+New section per PLAN.md 9.5 item 3, placed directly after the Actual-vs-Budget cards: hand-rolled SVG waterfall (FY Budget → YTD Actual → Secured backlog → Weighted pipeline → Remaining gap) with Revenue/GP toggle (same client-side pattern as the AVB chart), coverage KPI tiles, unrecognized-status footnote, and the fy_est cross-check footnote. Colors per 9.5. Sections must degrade gracefully (render an explanatory note) when the snapshot lacks the `bridge`/`coverage` blocks.
+
+**Validation command:**
+```
+python -m pytest tests/test_dashboard.py -q
+```
+**Expected result:** all pass, including: output contains the waterfall section for a phase-2 snapshot; output contains a graceful note (not a crash) for a legacy snapshot dict without the new blocks; still no `http://`/`https://` in the HTML.
+
+---
+
+### Task 19 — Dashboard: GP-margin trend line, negative-GP KPI, annuity split
+**File:** `src/fdat/dashboard.py` (+ tests)
+
+Per PLAN.md 9.5 items 1, 5, 6: GP-margin % line on the existing trend chart with a right-hand % axis (color `#262579`, dashed, own toggle checkbox); KPI tile "Deals at negative GP (month)" showing current count and prior-month count (prior from history — pass it into `render_dashboard` via the existing series/budget_series pattern, not a new global); annuity vs project stacked bar (month Act + FY Est) labeled "keyword-classified (approximate)".
+
+**Validation command:**
+```
+python -m pytest tests/test_dashboard.py -q
+```
+**Expected result:** all pass.
+
+---
+
+### Task 20 — Dashboard: order-intake section + exco reordering + demotions
+**File:** `src/fdat/dashboard.py` (+ tests)
+
+- Order-intake bars (revenue per period, deal-count annotation); when <2 snapshots render the "insufficient history" note (PLAN.md 9.2.3).
+- Reorder sections to the 9.5 sequence.
+- Demote inventory to the single KPI tile defined in 9.5.7 (remove the two aging cards from the dashboard).
+- Remove the duplicate `_wip_overdue_bars` card (keep the table and WIP KPI tiles).
+
+**Validation command:**
+```
+python -m pytest tests/test_dashboard.py -q && python -c "
+from tests.fixtures.make_synthetic_workbook import build_workbook
+import tempfile, os
+# smoke: full render on synthetic data, then manual browser check
+"
+```
+**Expected result:** tests pass. Then run the full CLI against the synthetic fixture, open the dashboard in a browser, and confirm: new section order, waterfall renders, no console errors. Note the manual check outcome in the commit message.
+
+---
+
+### Task 21 — Board pack: exec-summary bullets + bridge slide
+**Read skill: `pptx` first.** **File:** `src/fdat/boardpack.py` (+ tests in `tests/test_boardpack.py`)
+
+Per PLAN.md 9.6: extend exec-summary bullets (cap 6) and insert the "Path to Full Year" slide (matplotlib waterfall PNG, brand colors, layout index 3 `Content Slide 1`) after the P&L summary slide. Keep the fail-loudly layout-name check. Move the detailed inventory aging table to the appendix slide.
+
+**Validation command:**
+```
+python -m pytest tests/test_boardpack.py -q
+```
+**Expected result:** all pass; slide count assertion updated for the added slide.
+
+---
+
+### Task 22 — Full suite + end-to-end run
+**Validation command:**
+```
+python -m pytest tests/ -q && python -m src.fdat.cli run tests/fixtures/synthetic_sample.xlsx --period 2024-01 --history-dir /tmp/fdat_p2_hist --template templates/board_template.pptx --output-dir /tmp/fdat_p2_out
+```
+**Expected result:** suite green; CLI writes dashboard + boardpack with no traceback. STOP rule applies.