diff --git a/P2-RENDER-DESIGN.md b/P2-RENDER-DESIGN.md new file mode 100644 index 0000000..46a4a60 --- /dev/null +++ b/P2-RENDER-DESIGN.md @@ -0,0 +1,274 @@ +# P2 — Headless native map renderer for `ae.report`: design + work-breakdown + +**Goal (Eu, 2026-07-08).** Render the seasonal report's antigenic maps **without launching +kateri per map** (faster reports), **improve AD fidelity**, and stay **kateri-compatible** — +consume the *same on-chart styling* the report already feeds kateri, so the renderer is a +drop-in for the report's map step. kateri stays available (interactive drag/relax). + +This document is the Phase-1 spike output: it (a) writes down the *rendering contract*, (b) +records what the POC proved, (c) proposes the architecture, and (d) gives an **independent, +parallelisable** work-breakdown for the full build. It supersedes and extends +`P2-RENDER-SPIKE-PLAN.md` (A7's paper analysis) with a working headless POC and the key +de-risking findings. + +--- + +## 0. Executive summary + +- **The styling can be consumed headlessly — proven.** A ~230-line headless POC + (`tools/p2-render-spike/poc_render.py`) reads a real report chart's on-chart styles + (`c["R"]` named styles + `c["p"]` base plot-spec), resolves the selected front style, + applies per-point fill/outline/size/shape/order, draws the title, vaccine labels, and + viewport, and produces a map **recognisably identical in content** to both the kateri + golden and the AD reference for the same map (same two clusters, full clade palette, same + vaccine labels). This is the core unknown A7 left open (A7 only showed the *current* + `map-draw` pipeline ignores styles → ~0% content parity). +- **The C++ head-start is larger than A7 stated.** `cc/chart/v3/chart-import.cc` **already + fully parses `c["R"]`** into `cc/chart/v3/styles.hh`'s `semantic::Styles` + (modifiers, selectors, title, legend, serum circle, serum coverage). The data model *and* + the JSON reader exist and round-trip through Python (`chart.styles()`). Only the + **style resolver + drawing application** are missing from `cc/map-draw/draw.cc`. +- **Fidelity (normalised RMSE, one representative map, coarse):** headless-vs-kateri 0.29, + headless-vs-AD 0.28, **kateri-vs-AD 0.23**. The kateri-vs-AD 0.23 is a *framing/legend/scale + floor* between differently-exported maps — the POC sits just above it, and the excess is the + known, bounded tail (exact viewport, legend box, grid, fonts/AA). Pixel-RMSE is **not** a + clean metric until viewport + page geometry are matched; the visual montage is the honest + evidence at this stage. +- **Recommendation: GO, staged, multi-phase (multi-agent).** The spine is proven and the data + model + reader + drawing primitives already exist. Gate the commitment on milestone **A** + (one plain by-clade map < ~2% pixel-diff vs the kateri golden once the viewport convention + is matched). The sub-features below are largely independent and can be parallelised. + +--- + +## 1. The rendering contract (the spec the renderer implements) + +### 1.1 What the report feeds kateri + +The report never sends a "how to draw" plot spec. `ChartModifier` +(`py/ae/report/chart_modifier.py`) bakes onto the chart: + +1. **Semantic attributes** on antigens/sera (`ae.semantic.*`): clade list (`C`), reference + (`R`), passage (`p`), continent/country (`C9`/`c9`), older-than (`o6m`/`o12m`), + new-compared-to-previous (`new`), vaccine (`V`), serology, sequenced. +2. **Named styles** in the chart's `c["R"]` block (C++ `semantic::Styles`). + +`py/ae/utils/kateri.py` then transports over a length-prefixed Unix socket (4-byte code + +4-byte length + 4-byte-padded payload): `send_chart` (`CHRT`), `set_style(name)` (`COMD`), +`get_pdf(style,width,square,viewportSize)` (`COMD` → `PDFB` bytes). kateri resolves the +named style, applies it, draws title/legend/grid/serum-circles, frames by the viewport, and +returns PDF bytes. The report's map step is exactly: **pick a style name → get a PDF**. That +is the whole surface the headless renderer must replace. + +### 1.2 On-chart representation (verified against a real styled chart) + +`c["R"]` is an object: `name → style`. Two kinds of style: + +**Front (composite) style** — the thing a map is rendered as. Carries: +- `z` : integer priority (inter-style draw order). +- `A` : ordered list; each entry `{ "R": }` is a **reference** that composes a + background style (resolved by following the reference). +- `L` : legend flags `{ "-": shown(bool), "C": add_counter(bool) }`. +- `T` : title `{ "B": box{ "o": origin, "O": [dx,dy], padding/border/bg }, "T": text{ + "t": text, "f": face, "W": weight, "S": slant, "s": size, "c": colour, "i": interline } }`. + +**Background style** — a list of selector→point-style rules, plus optional viewport: +- `z` : priority. +- `V` : `[x, y, w, h]` viewport (present on the `-reset` family). +- `A` : ordered modifier list. Each modifier: + - `T` : **selector** object — `{C: clade}` (matches if clade ∈ antigen's `C` list), + `{R: true}` reference, `{V: true}` vaccine, `{p: passage}`, `{"!i": index}` by point + index; empty ⇒ all candidates. + - `A` : select — `1`/truthy = antigens only, `0`/false = sera only, absent = both. + - point-style fields (shared with `c["p"]`): `F` fill, `O` outline, `o` outline-width, + `S` shape (`C`ircle/`B`ox/`T`riangle/`E`gg/`U`glyegg), `s` size, `r` rotation, + `a` aspect, `-` hidden (`false` ⇒ shown), `l` label `{p:[dx,dy], t:text, s:size}`. + - `D` : drawing order — `"r"` raise, `"l"` lower. + - `L` : legend row `{ p: priority, t: text }`. + - `CI` : serum-circle spec; `SC` : serum-coverage spec (see `styles.hh`). + +**Base plot-spec `c["p"]`** (kateri's starting point before named-style modifiers): +`P` = list of point styles, `p` = per-point index into `P`, `d` = draw order. Same field +letters. This supplies the default appearance (base greys, shapes, sizes) that the named +style then overrides. + +**Resolution algorithm** (kateri's, reproduced in the POC): start from `c["p"]` per-point +styles; resolve the selected front style by walking its `A` list, recursively expanding each +`{R:name}` reference into that background style's modifiers (picking up its `V` viewport), +producing a **flat, ordered modifier list**; apply each modifier to the points its selector +matches (later modifiers win; `D:"r"` moves matched points to the end of the draw order); +collect legend rows; apply the front style's own `T`/`L`. Undefined references are tolerated +(e.g. a `-new-*` ref absent on a chart with no previous). Draw in draw order; then title; +then legend; frame by the viewport. + +The C++ structs for all of this already exist (`styles.hh`) and the reader already +populates them (`chart-import.cc` `read_semantic_plot_specification` → `…_style` → +`…_style_modifier`). **The resolver above is the missing piece** (currently only kateri/Dart +implements it). + +### 1.3 AD heritage (fidelity target) + +Where kateri diverges from AD, match **AD**. AD's renderer is +`~/AC/eu/AD/sources/acmacs-draw` + `acmacs-map-draw` (`draw.cc`, `point-style-draw.cc`, +`map-elements*.{cc,hh}`, `labels.*`, legend + viewport code). The revived `cc/map-draw` +already reproduced AD *chain* maps to <1% px-diff, so the cairo/viewport/point-shape layer is +AD-calibrated. The POC confirmed the report's kateri golden and the AD reference for the same +map are near-identical in content (same clades/counts/labels), so for the by-clade maps +"match kateri" and "match AD" coincide; the fidelity strategy only needs an explicit AD tie- +breaker for details where they differ (grid style, legend metrics, label auto-placement). + +--- + +## 2. POC — what was actually built and proven + +`tools/p2-render-spike/poc_render.py` (Python + Pillow, stdlib elsewhere), run headlessly: + +``` +python3.14 poc_render.py +``` + +It implements §1.2 end-to-end: colour parsing (hex/#AARRGGBB/named/`gray80`/`:bright`), +base plot-spec extraction, selector matching against semantic attributes, recursive front- +style resolution, per-point application, draw-order raising, title, vaccine labels, legend- +row collection, and a Pillow rasteriser (circle/box/triangle/egg-approx). Reads the chart +**read-only** from a private report dir; no WHO data enters the repo (images stay in the +scratchpad). + +**Result.** For the representative `clades` main map it draws the correct two clusters, the +**full clade palette matching kateri and AD**, base grey cloud, open-box sera, open-circle +references, the title, and all vaccine labels. See the 3-way montage (POC | kateri | AD) in +the scratchpad. This validates the whole spine: style read → selector → point-style → +title → legend → viewport, headless, on a real report chart. + +**What the POC did NOT do (⇒ the fidelity tail):** exact viewport/aspect match to kateri +(it auto-frames to the data bbox), the legend *box* with per-row counts, the background +unit grid + border, exact fonts/anti-aliasing/PDF page geometry, serum circles/coverage, +time-series / continent / serology / info-map style families, label auto-placement (overlap +avoidance). These are the §4 tasks. + +### 2.1 Vertical-mirror resolution + +The first POC pass applied a Y-flip (`device_y = H - (y-vy)/vh*H`) and came out vertically +mirrored vs kateri. **Fix: apply no Y-flip** — `device_y = (y - vy)/vh * H`. The projection +layout `c["P"][0]["l"]`, after the stored 2×2 transform `t` is applied +(`x' = t0·x + t1·y`, `y' = t2·x + t3·y`), is already in a **y-increases-downward screen +coordinate system**; kateri maps world→device directly with no inversion. With the flip +removed the POC matches kateri's orientation exactly (diverse cluster top, blue cluster +bottom). The native renderer must likewise treat post-transform layout Y as screen-down. + +### 2.2 Viewport convention (open item, must be nailed in milestone A) + +The stored `-reset` viewport `V=[x,y,w,h]` did **not** frame the post-transform layout in the +POC (data y-range fell outside the viewport y-range), so the POC auto-frames instead. kateri +frames correctly, so kateri applies an additional **recenter** between the transform and the +viewport (cf. `export_mapi_for_signature_pages` in `chart_modifier.py`, which combines +kateri's `native`, `used`, and `native_center` to recover an absolute viewport — evidence +kateri carries a native-center offset). Milestone A must reproduce this exact transform + +recenter + viewport chain so the native frame equals kateri's; it is the single most +important fidelity step (everything keys off it) and the biggest residual in the POC numbers. + +--- + +## 3. Architecture — native renderer as a kateri-compatible drop-in + +### 3.1 Component + +Add a **semantic-style interpreter** layer to `cc/map-draw` on top of its existing cairo / +viewport / point-shape scaffolding, plus a new entry point that takes a **chart + style +name** (mirroring `set_style` + `get_pdf`) rather than the chains `DrawSettings`: + +``` +export_styled_map(chart, projection_no, style_name, width, output) // C++ + pybind +``` + +Pipeline inside: `resolve_style(chart.styles(), c["p"], style_name)` → flat modifier list + +viewport + title + legend (the §1.2 algorithm, in C++, reusing `semantic::Styles` already +parsed by the importer and `SelectedAntigens/Sera` for selector matching) → apply to a +per-point render model → draw via `cc/draw/cairo-surface` (points, grid/border, title, +legend, serum circles) → PDF/PNG. The chains `export_map` stays as-is (fixed AD-chains +pipeline); the new path is additive. + +### 3.2 Where it plugs into `ae.report` + +The report's map step is "select style → get PDF" via `kateri.communicator` +(`set_style`/`get_pdf`) in the `style`/`export` path (`ae.report.commander`). Introduce a +small **renderer seam** — a `MapRenderer` interface with two implementations: + +- `KateriRenderer` — current behaviour (launch kateri, socket, `set_style`/`get_pdf`). +- `NativeRenderer` — call `ae_backend.map_draw.export_styled_map(chart, style, width)` + in-process; no subprocess, no socket, Linux-capable. + +Select via a config flag / env (`AE_REPORT_MAP_RENDERER=native|kateri`, default `kateri` +until milestone D proves parity, then flip). This keeps kateri as a **fallback** and as the +**interactive** tool (drag-adjust / relax — `handle_relax`, `get_moved_points`), which the +native batch renderer deliberately does not replace. + +### 3.3 AD-fidelity strategy + +1. Match **viewport/transform/recenter** first (§2.2) — the frame must equal kateri's. +2. Point shapes/sizes/outlines/greys from `c["p"]` + modifiers (POC already correct in + content) — calibrate pixel size against the AD/kateri golden. +3. Grid + border, legend box metrics, title box metrics, fonts (Helvetica weights/slants), + PDF page geometry: take AD (`acmacs-draw`/`acmacs-map-draw`) as the tie-breaker where it + differs from kateri; the chains `map-draw` revival already matched AD here. +4. Per-map-family details (serum circles theoretical/empirical, coverage within/outside, + time-series sizing) from the modifier specs in `styles.hh`. + +Fidelity harness: rasterise native + kateri golden with `pdftoppm` at equal size and +`magick compare -metric RMSE/AE` + a high-zoom montage, over the figure matrix (subtypes × +labs × style families). Reuse the chains map-draw pixel-diff helpers. + +--- + +## 4. Work-breakdown (independent sub-features — parallelisable) + +Milestone **A** is the go/no-go spine and must land first. After A, the rest are largely +independent and can be handed to separate agents in parallel; dependencies noted. + +| ID | Sub-feature | Depends on | Size | Hard parts | +|----|-------------|-----------|------|-----------| +| **A** | **Spine + go/no-go.** C++ `export_styled_map`: style resolver (front→background composition, flat modifier list), selector engine (reuse `SelectedAntigens/Sera`), per-point fill/outline/width/size/shape/order, **transform+recenter+viewport exact match** (§2.2), title, plain legend. Target: one plain by-clade map < ~2% px-diff vs kateri golden. | reader (done) | **M** | **Viewport recenter convention** (the key risk); flat-resolution order vs kateri; colour model (`:bright`, `#AARRGGBB`, `gray80`). | +| **B** | Legend box | A | S–M | Per-row counts (`legend_counter` ⇒ count matched points), zero-count rows, box origin/padding, row point-size/text-size metrics matching kateri/AD. | +| **C** | Title + fonts + PDF page geometry | A | S–M | Helvetica face/weight/slant, interline, box origin/offset, multi-line titles; PDF point size = width; AA parity. | +| **D** | Background grid + border | A | S | AD `BackgroundBorderGrid` spacing/colour; cheap, isolate early for fidelity. | +| **E** | Remaining selectors + composed styles | A | M | reference/passage/continent/older-than/new/serology/vaccine selectors; `-reset`/`-new`/`-vaccines`/`-pale`/`-o6m/12m` composition; **info-** maps (blank title, no legend). Mostly selector coverage — low risk once A's engine exists. | +| **F** | Serum circles | A | M | Empirical vs theoretical radius, fold, dash, angle radius-lines, per-passage outline/fill, fallback radius. Geometry-heavy; AD `map-procrustes`/serum-circle code is the reference. | +| **G** | Serum coverage | F | M | within/outside point restyle by fold; `-sco-*`/`sc-*` front styles; per-serum map matrix. | +| **H** | Time-series / continent / serology / pale families | E | M | Composed multi-reference styles; per-month `ts-*`; old/new sizing. Volume, not novelty. | +| **I** | Label auto-placement | A,C | **M–L** | Vaccine/serology labels currently carry explicit offsets (`l.p`) so baseline is easy; **overlap-avoidance / leader lines** to match AD is the genuinely hard, iterative part. | +| **J** | `ae.report` renderer seam + flag | A | S | `MapRenderer` interface, `NativeRenderer`, `AE_REPORT_MAP_RENDERER`; keep kateri default + fallback. Wire producer **and** consumer (a feature isn't done until end-to-end). | +| **K** | Fidelity harness + figure-matrix sign-off | B–J | **M–L**, iterative | The long tail: per-family px-diff to <~1–2%, document irreducible AA diffs, flag any map that can't match and why. | + +**Stays on kateri (do not port):** interactive drag-adjust + live relax animation +(`RLAX`/`LAYT`/`get_moved_points`/`handle_relax`) and the operator GUI. The native renderer +is the **batch** map engine only. + +**Downstream (out of P2 scope, enabled by it):** single-canvas signature pages — draw the +`tal-draw` tree and the per-section `map-draw` maps on one `CairoSurface` (today composed via +`pdfjam`/`pdflatex` in `signature_page.py`/`section_maps.py`). Defer until A–E land. + +--- + +## 5. Recommendation + +**GO, staged, multi-phase (multi-agent).** The architecture matches the author's stated +intent (one fast C++ JSON→image/PDF engine, kateri as thin/interactive client), the data +model + JSON reader + AD-calibrated drawing layer already exist, and the POC proves the +styling is consumable headlessly with content parity to both kateri and AD. + +- **Next step:** milestone **A** as a single focused agent-session. Success = the plain + by-clade map frames and colours like the kateri golden (< ~2% px-diff), which requires + cracking the viewport recenter convention (§2.2) — the one real risk. **If A cannot reach + parity within ~2 sessions, that is the no-go signal**: keep kateri for report maps and + `map-draw` chains-only. +- **After A:** fan out B–J to parallel agents (independent per the table), then converge on + the K harness. Total ≈ a multi-week C++ effort comparable to the chains `map-draw` revival, + but front-loaded risk is small because the reader/model/primitives are done — the cost is + the fidelity long tail (viewport, legend/title metrics, fonts, label placement), not new + architecture. + +**Honesty on the POC:** it is a Python/Pillow *content* proof, not a pixel-faithful renderer. +It does not match kateri's viewport, legend box, grid, or fonts, and its RMSE numbers are +dominated by those gaps (the kateri-vs-AD 0.23 floor shows RMSE is uninformative until +framing is matched). What it *does* prove — the previously-open question — is that the report's +on-chart styling can be parsed and applied headlessly to reproduce the report map's content. diff --git a/P2-RENDER-SPIKE-PLAN.md b/P2-RENDER-SPIKE-PLAN.md new file mode 100644 index 0000000..cbb8ced --- /dev/null +++ b/P2-RENDER-SPIKE-PLAN.md @@ -0,0 +1,175 @@ +# P2 rendering spike — can `map-draw` become the canonical `ae.report` figure engine? + +**Spike goal.** Decide whether the headless C++ `map-draw` renderer (subsystem #1, revived +for the Linux whocc-chains batch path) can replace **kateri** (a Dart/Flutter macOS app, +driven over a Unix socket) as the engine that renders antigenic-map figures for +`ae.report`. The original author's intended architecture is a *single fast C++ engine that +reads JSON → image/PDF*, driven from Python and a thin GUI; kateri was a GUI experiment. + +**Status:** spike complete. Gap characterised; one representative figure prototyped and +pixel-diffed against its kateri golden. Recommendation below is **conditional GO, staged**. + +--- + +## 1. The two input paths + +### 1a. What `ae.report` sends to kateri (today's report figure path) + +The report never hands kateri a "plot spec" describing *how to draw*. It hands kateri a +**fully self-describing chart** and a **style name**: + +- `ChartModifier` (`py/ae/report/chart_modifier.py`) writes onto the chart: + 1. **Semantic attributes** on antigens/sera — clade, reference, passage, continent/country, + older-than, new-compared-to-previous, vaccine, serology (`ae.semantic.*`). + 2. **Named styles** in the chart's `c["R"]` block (`ae_backend` `SemanticStyles`), e.g. + `-reset`, `-clades`, `-vaccines`, `clades`, `clades-6m/-12m`, `info-clades`, `serology`, + `-o6m-grey`, `-continent`, `-pale`, `ts-YYYY-MM`, serum-circle / serum-coverage styles. + A **front style** (e.g. `clades`) *composes* background styles by reference + (`A: [{R:"-reset"}, {R:"-clades"}, {R:"-new-1"}, {R:"-vaccines"}]`) and carries a + **title** (box origin/offset + text font/weight/slant/size/colour/interline), a + **legend** flag block, and a **viewport/zoom** reset (`L`). A background style + (e.g. `-clades`) is a **list of selector→point-style rules**: `{T: selector, F: fill, + O: outline, D: drawing-order, A: alpha, L: {p: legend-priority, t: legend-text}}`. +- Transport (`py/ae/utils/kateri.py`): `send_chart(chart)` → `set_style(name)` → + `get_pdf(style=name)`. kateri resolves the named style, applies the per-selector point-style + modifications, draws title/legend/serum-circles/grid, frames by the stored viewport, and + returns PDF bytes. + +The C++ structs for this style model **already exist in ae** at `cc/chart/v3/styles.hh` +(`Title`, `Legend`, `box_t`, `text_t`, `point_style_fow_t`, `serum_circle_style_t`, +`serum_coverage_style_t`, `Selector`) — but only kateri (Dart) currently *interprets* them. + +### 1b. What `map-draw` consumes + +`ae_backend.map_draw.export_map(ace, output, projection_no, size, reorient_master, mapi, +coloring, marks, title, legend, labels, serum_circles)` / the `map-draw` CLI. It is a +**fixed-pipeline reproduction of AD's chains-202105 `make_map`**, *not* a style interpreter: + +- Grey base points with **AD-chains hardcoded** shapes/sizes (`outline #D0D0D0`, test d10 / + reference d15 (open circle) / serum d15 (open box)) — set in `draw.cc`, not read from the chart. +- Clade colour re-derived from an **external `clades.mapi` DSL file** + `coloring_key`, after + **populating clades from seqdb** — *not* from the chart's baked semantic attributes/styles. +- `mark_recent_layer`, `mark_vaccines` (vaccine strains read at runtime from + `acmacs-data/semantic_vaccines.py`), a **stress-value** title, a mapi-derived clade legend, + a **self-computed bounding-ball** viewport (or `reorient-master` alignment), opt-in + hardcoded 2-fold serum circles. + +**Confirmed by inspection:** `cc/map-draw/draw.cc` has **zero** references to `chart.styles()` +(`c["R"]`) or the legacy plot-spec (`c["p"]`). It ignores the entire report styling system. + +### 1c. The gap (enumerated) + +| # | Report figure needs (kateri does it) | `map-draw` today | +|---|---|---| +| 1 | Resolve named front styles + background-style composition + priority (`c["R"]`) | **missing entirely** | +| 2 | Per-point styles from the legacy plot-spec (`c["p"]`) | **ignored** (AD-chains hardcoded defaults) | +| 3 | Per-selector fill/outline/outline-width/size/shape/alpha/drawing-order | wrong (fixed greys + fixed sizes) | +| 4 | Arbitrary title: text + box origin/offset + font face/weight/slant/size/colour/interline | only a stress number, top-left | +| 5 | Semantic legend: rows (priority+text), counter, box, point-size, zero-count rows | only a mapi-derived clade legend | +| 6 | Selectors beyond clade: reference, passage, continent, older-than, new-compared-to, serology, vaccine | only clade + reference + serum | +| 7 | Stored/zoom viewport from the style (`L`) → exact report framing | self-computed bbox → **framing/zoom mismatch** | +| 8 | Serum circles from style: fold, theoretical/empirical, dash, angles, radius outline | hardcoded 2-fold empirical only | +| 9 | Serum-coverage (within/outside) styling | missing | +| 10 | Time-series / pale / continent / serology composed styles | missing | +| 11 | Vaccine marks driven by the chart's semantic vaccine attribute + style | different source (runtime `semantic_vaccines.py`) | +| 12 | Font matching to kateri (Helvetica, weights/slants) + PDF page geometry | partial / not matched | + +**Root characterisation:** `map-draw` is a *fixed AD-chains pipeline*; the report needs a +*semantic-style interpreter*. Closing the gap = porting kateri's rendering logic (Dart) into +C++ on top of `map-draw`'s existing cairo/viewport/point scaffolding. + +--- + +## 2. Prototype (what was actually validated) + +One representative report figure: the `clades` main map of a small H1 report chart +(8 antigens × 7 sera). Rendered through `map-draw` (`--no-populate --no-marks --no-vaccines`) +and pixel-compared to the **existing kateri golden** `out.1.clades.pdf` for the same chart. +(No kateri re-run needed — the golden was already on disk. Real chart + renders kept in the +scratchpad, never in the repo.) + +- **RMSE = 0.155 (15.5%).** +- **Visual (montage):** kateri shows the report title, clade-coloured antigens, green + vaccine dots with strain labels, and a full clade legend with counts, framed by the stored + zoom viewport. `map-draw` shows an all-grey map (reference = open circles, sera = open + boxes) with a stress-number title, **no** colour / vaccines / legend / report title, at a + **different zoom/framing**. Point *positions* correspond (same optimisation), but scale + differs because the viewports differ. +- **Effective report-figure parity of `map-draw` as-is: ~0%.** The 15.5% RMSE understates it + because most of the canvas is white background; every foreground element differs. + +This is the expected result and confirms the gap is architectural, not cosmetic: +`map-draw` cannot approximate a report figure by tweaking flags — it lacks the style engine. + +Repro: `tools/p2-render-spike/compare.sh `. + +--- + +## 3. Parity + effort plan + +Realistic path: add a **semantic-style interpreter** to `map-draw` — read `c["R"]` + `c["p"]`, +resolve/compose front styles, apply per-selector point styles, and draw title/legend/viewport/ +serum-circles from the style. Reuse what already exists: + +- `cc/chart/v3/styles.hh` already **models** the style structs (big head-start — the schema is + done; it needs a JSON reader + an interpreter, not a new data model). +- `cc/chart/v3` selection (`SelectedAntigens`/`SelectedSera`) already matches semantic + attributes — reuse for selectors. +- `map-draw` already has cairo primitives, the viewport/transform, point shapes, and + title/legend/serum-circle drawing code — the *drawing* layer is largely there; the + *style-interpretation* layer is ~0%. +- whocc-chains proved `map-draw` can reach **<1% pixel diff** vs an AD golden — the fidelity + bar is achievable. + +Staged milestones (each ends with a pixel-diff vs a kateri golden across the report set): + +| Stage | Scope | Est. (agent-pace) | +|---|---|---| +| P2-A | Style reader for `c["R"]`/`c["p"]` + front-style resolver + selector engine; apply per-point fill/outline/size/shape/order; draw title (arbitrary text/box/font) + semantic legend + **viewport from style**. Target: `clades` main map < 1% vs kateri golden. | **M** (1–2 sessions) | +| P2-B | Remaining selectors (reference/passage/continent/older-than/new/serology/vaccine) + `-vaccines`/`-reset`/`-new` composition + info-/6m/12m variants. | **M** | +| P2-C | Serum circles + serum coverage from style (fold/theoretical/empirical/dash/angles/radius); time-series / pale / continent / serology styles. | **M** | +| P2-D | Font + PDF-geometry parity with kateri; fidelity harness over the full figure matrix (subtypes × labs × styles); iterate to < 1%. | **M–L**, iterative | + +**Total ≈ a multi-week C++ effort (several agent-sessions)** — comparable to or somewhat larger +than the whocc-chains `map-draw` revival, because the target is *arbitrary report styling* +parity, not one fixed pipeline. The `styles.hh` head-start and existing drawing layer are the +main risk-reducers; the long tail (exact fonts, legend/title box metrics, viewport rounding, +per-style-family quirks) is where iteration time goes. + +**What is validated vs estimated:** the gap analysis and the ~0% as-is parity are +**validated** (inspection + prototype). The stage effort is **estimated** by analogy to the +chains revival and by the fact that the data model already exists. + +--- + +## 4. Recommendation + +### P2 (report figure engine): **conditional GO — staged.** + +The architecture case is strong and matches the author's intent: one fast, Linux-capable C++ +engine reading JSON → PDF, no Dart/macOS/socket dependency, no separate GUI process in the +batch path. The work is substantial but **bounded and well-scaffolded** (`styles.hh` models +the styles; the drawing layer exists; chains proved <1% fidelity is reachable). + +De-risk by gating on **P2-A**: build the style interpreter far enough to reproduce the plain +`clades` main map **< 1% pixel-diff vs the kateri golden**. That single milestone exercises the +whole spine (style read → selector → point-style → title → legend → viewport). If P2-A lands +cleanly, commit to P2-B..D and retire kateri from the report figure path. **If P2-A cannot +reach < 1% within ~2 sessions, that is the no-go signal** — kateri stays the report renderer +and `map-draw` remains the chains-only tool. + +### sigp (AD-faithful maps + tree on one C++ canvas): **feasible, but downstream of P2.** + +`map-draw` and `tal-draw` both build on the shared `cc/draw/cairo-surface` primitives, and AD +historically drew the tree and the section maps on a **single canvas**. Today ae composes a +`tal-draw` tree PDF with **kateri** map PDFs via `pdfjam`/`pdflatex` +(`py/ae/tal/signature_page.py` + `section_maps.py`; the latter's own docstring notes "ae has no +single-canvas renderer"). Once P2 gives `map-draw` a report-grade map renderer, a true +single-canvas sig page becomes achievable: draw the tree (tal-draw code) and the per-section +maps (map-draw code) onto one `CairoSurface`, positioned by the sig-page layout, with +per-section colouring that `section_maps.py` already computes. + +**Verdict:** sigp single-canvas is a clean architectural win but is **contingent on P2** (needs +the map renderer first) **plus a compositor** (shared coordinate space, section→map viewport, +page layout). Recommend: **defer sigp until P2-A/B prove the map engine**, then scope it as a +follow-on. It should not gate the P2 decision. diff --git a/cc/draw/cairo-surface.cc b/cc/draw/cairo-surface.cc index f177bca..1e36894 100644 --- a/cc/draw/cairo-surface.cc +++ b/cc/draw/cairo-surface.cc @@ -10,6 +10,13 @@ namespace ae::draw { + // kateri renders map text 2% larger than the nominal size (PdfGraphics.fontScaleToMatchCanvas, + // draw_on_pdf.dart) so its PDF text matches its on-screen canvas. The report's golden PDFs carry + // that factor, so the semantic-style text primitives (text_font / helvetica text_size) apply it + // too — both the drawn glyphs and the metrics the callers size their boxes from — to land on the + // golden's glyph pixels rather than ~2% short. + static constexpr double kFontScaleToMatchCanvas = 1.02; + static inline void set_source(_cairo* cr, Color color) { cairo_set_source_rgba(cr, color.red(), color.green(), color.blue(), color.alpha()); @@ -92,6 +99,18 @@ namespace ae::draw cairo_new_path(context_); // discard the preserved path if we didn't stroke } + void CairoPdf::arc(double cx, double cy, double radius, double start_angle, double end_angle, Color outline, double outline_width) + { + if (outline_width <= 0.0 || outline.is_transparent()) + return; + constexpr double twelve_oclock = -std::numbers::pi / 2.0; + cairo_new_path(context_); + cairo_arc(context_, cx, cy, radius, twelve_oclock + start_angle, twelve_oclock + end_angle); + set_source(context_, outline); + cairo_set_line_width(context_, outline_width); + cairo_stroke(context_); + } + void CairoPdf::square(double cx, double cy, double side, Color outline, double outline_width, Color fill) { const double half = side / 2.0; @@ -123,6 +142,30 @@ namespace ae::draw cairo_stroke(context_); } + void CairoPdf::egg(double cx, double cy, double size, Color outline, double outline_width, Color fill) + { + // Reproduces kateri's _drawShape egg (draw_on_pdf.dart): two cubic beziers between the + // top (0, -r) and bottom (0, +r) apexes, y growing downward (matches our device space, + // no Y-flip). r = size/2. + const double r = size / 2.0; + cairo_new_path(context_); + cairo_move_to(context_, cx + 0.0, cy + r); + cairo_curve_to(context_, cx + r * 1.4, cy + r * 0.95, cx + r * 0.8, cy - r * 0.98, cx + 0.0, cy - r); + cairo_curve_to(context_, cx - r * 0.8, cy - r * 0.98, cx - r * 1.4, cy + r * 0.95, cx + 0.0, cy + r); + cairo_close_path(context_); + if (!fill.is_transparent()) { + set_source(context_, fill); + cairo_fill_preserve(context_); + } + if (outline_width > 0.0 && !outline.is_transparent()) { + set_source(context_, outline); + cairo_set_line_width(context_, outline_width); + cairo_stroke(context_); + } + else + cairo_new_path(context_); + } + void CairoPdf::filled_triangle(double x0, double y0, double x1, double y1, double x2, double y2, Color fill) { cairo_new_path(context_); @@ -240,9 +283,58 @@ namespace ae::draw } } - std::pair CairoPdf::text_size(std::string_view utf8, double font_size) + void CairoPdf::text_font(double x, double y, std::string_view utf8, double font_size, Color color, bool bold, bool italic, + double halo_width, Color halo_color) + { + // Baseline-origin anchor (matches kateri drawString at (origin.dx, origin.dy)): (x, y) is + // the pen origin — baseline at y, first glyph's pen position at x (ink starts at + // x + left-side-bearing, exactly as the golden's drawString does). + const std::string str{utf8}; + // kateri renders Latin1 text in Helvetica (Type1); use the same face so the native + // render's title/legend/labels match the golden's glyph shapes/metrics. On this + // toolchain cairo's toy "Helvetica" resolves (via fontconfig) to the same face poppler + // substitutes for the golden PDF's non-embedded base-14 Helvetica, so an embedded + // subset rasterises pixel-identically — the remaining tail is glyph *placement*. + cairo_select_font_face(context_, "Helvetica", italic ? CAIRO_FONT_SLANT_ITALIC : CAIRO_FONT_SLANT_NORMAL, bold ? CAIRO_FONT_WEIGHT_BOLD : CAIRO_FONT_WEIGHT_NORMAL); + cairo_set_font_size(context_, font_size * kFontScaleToMatchCanvas); + // kateri's PdfGraphics.drawString anchors the glyph *pen origin* (baseline-left) at the + // supplied point — it does NOT shift by the first glyph's left side-bearing. Match that + // (a prior x_bearing subtraction pushed every string ~1px left of the golden). + cairo_move_to(context_, x, y); + if (halo_width > 0.0) { + // kateri addPointLabel halo: stroke the glyph outlines in the halo colour first (round + // joins for a smooth band, scaled with the glyph), then fill the glyphs on top so the + // label reads over the point cloud (drawString stroke pass under the fill). + cairo_text_path(context_, str.c_str()); + set_source(context_, halo_color); + cairo_set_line_width(context_, halo_width * kFontScaleToMatchCanvas); + cairo_set_line_join(context_, CAIRO_LINE_JOIN_ROUND); + cairo_stroke_preserve(context_); + set_source(context_, color); + cairo_fill(context_); + } + else { + set_source(context_, color); + cairo_show_text(context_, str.c_str()); + } + } + + std::pair CairoPdf::text_size(std::string_view utf8, double font_size, bool helvetica) { const std::string str{utf8}; + if (helvetica) { + // Match kateri's PdfGraphics.textSize so callers (title box, legend rows) lay text out + // exactly where the golden has it: width = the font's *advance* width (kateri uses the + // base-14 AFM metrics.width, i.e. the pen advance, not the ink extent), height = the + // em size itself (kateri hard-codes textSize height to 1.0*fontSize, not the 1.156 line + // height nor the ink height). Both scaled by fontScaleToMatchCanvas, as kateri does. + cairo_select_font_face(context_, "Helvetica", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); + const double scaled = font_size * kFontScaleToMatchCanvas; + cairo_set_font_size(context_, scaled); + cairo_text_extents_t ext; + cairo_text_extents(context_, str.c_str(), &ext); + return {ext.x_advance, scaled}; + } cairo_select_font_face(context_, "sans-serif", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); cairo_set_font_size(context_, font_size); cairo_text_extents_t ext; diff --git a/cc/draw/cairo-surface.hh b/cc/draw/cairo-surface.hh index 1a27f2b..3169063 100644 --- a/cc/draw/cairo-surface.hh +++ b/cc/draw/cairo-surface.hh @@ -42,8 +42,17 @@ namespace ae::draw // centre -> arc -> centre, so the wedge is closed. Transparent fill = outline only; // non-positive outline width / transparent outline skips the stroke. void sector(double cx, double cy, double radius, double start_angle, double end_angle, Color outline, double outline_width, Color fill); + // Open circular arc stroke (no fill, no radius lines to centre): the arc of the circle + // of `radius` centred at (cx, cy) from `start_angle` to `end_angle` (same clockwise-from- + // 12-o'clock convention as sector()). Used to build kateri's dashed serum circles, whose + // outline is a set of short arcs (unlike sector(), which closes the path to the centre). + void arc(double cx, double cy, double radius, double start_angle, double end_angle, Color outline, double outline_width); void square(double cx, double cy, double side, Color outline, double outline_width, Color fill); void triangle(double cx, double cy, double radius, Color outline, double outline_width, Color fill); // equilateral, point up + // Egg (kateri PointShape.egg): a closed two-bezier egg of the given `size` (bounding + // diameter) centred at (cx, cy), reproducing kateri _drawShape's control points so a + // native render matches the kateri golden exactly. Transparent fill = outline only. + void egg(double cx, double cy, double size, Color outline, double outline_width, Color fill); void filled_triangle(double x0, double y0, double x1, double y1, double x2, double y2, Color fill); // arbitrary filled triangle // Axis-aligned rectangle with its top-left corner at (x, y). Transparent fill = outline only. void rectangle(double x, double y, double width, double height, Color outline, double outline_width, Color fill); @@ -63,8 +72,17 @@ namespace ae::draw // strokes a halo (default white) behind the glyphs (see text()). void text_rotated(double x, double y, std::string_view utf8, double font_size, Color color, double angle_degrees, double halo_width = 0.0, Color halo_color = WHITE); + // Like text(center=false) but with selectable weight/slant (for the semantic-style + // title/legend, which carry helvetica bold/italic). (x, y) is the glyph-box top-left. + // halo_width > 0 strokes a halo (default white) behind the glyphs, so point labels stay + // legible over the point cloud (kateri's default point-label halo). halo_width is the full + // stroke width in font-size units (scaled internally to match the glyph), not a radius. + void text_font(double x, double y, std::string_view utf8, double font_size, Color color, bool bold, bool italic, + double halo_width = 0.0, Color halo_color = WHITE); // Measure a string at the given font size: returns {width, height} in device units. - std::pair text_size(std::string_view utf8, double font_size); + // `helvetica` selects the Helvetica face (matching text_font) so callers that draw with + // text_font size their boxes/rows from the same metrics; default keeps the sans-serif face. + std::pair text_size(std::string_view utf8, double font_size, bool helvetica = false); private: _cairo_surface* surface_{nullptr}; diff --git a/cc/map-draw/draw.hh b/cc/map-draw/draw.hh index f243221..55351de 100644 --- a/cc/map-draw/draw.hh +++ b/cc/map-draw/draw.hh @@ -54,6 +54,15 @@ namespace ae::map_draw // Render one projection of a chart to output (extension .png -> raster, else PDF). void export_map(const ae::chart::v3::Chart& chart, ae::projection_index projection_no, const std::filesystem::path& output, const DrawSettings& settings = {}); + // ------------------------------------------------------------------ + // P2 milestone A: headless render of a chart's on-chart *semantic* styling (c["R"] named + // styles + c["p"] base plot-spec), mirroring kateri's set_style + get_pdf. This is the + // report map path (by-clade etc.), additive to and independent of export_map above (the + // fixed AD-chains pipeline). `style_name` selects a front style in chart.styles(); + // `width` is the output width in device px / PDF points. Output extension picks backend + // (.png raster, else PDF). See cc/map-draw/STYLED-DRAW.md and P2-RENDER-DESIGN.md §1.2/§2.2. + void export_styled_map(const ae::chart::v3::Chart& chart, ae::projection_index projection_no, std::string_view style_name, double width, const std::filesystem::path& output); + // Procrustes render: draw `secondary` framed like `primary`, with arrows for common // points (threshold 0.3) and an "RMS: {rms:.4f}" title. See make_pc in chains chart.py. void export_procrustes(const ae::chart::v3::Chart& primary, ae::projection_index primary_projection, const ae::chart::v3::Chart& secondary, diff --git a/cc/map-draw/styled-draw.cc b/cc/map-draw/styled-draw.cc new file mode 100644 index 0000000..5d44798 --- /dev/null +++ b/cc/map-draw/styled-draw.cc @@ -0,0 +1,950 @@ +#include +#include +#include +#include +#include +#include +#include +#include + +#include "ext/fmt.hh" +#include "map-draw/draw.hh" +#include "draw/cairo-surface.hh" +#include "ad/color.hh" +#include "ad/color-hsv.hh" +#include "chart/v3/chart.hh" +#include "chart/v3/styles.hh" +#include "chart/v3/legacy-plot-spec.hh" +#include "chart/v3/layout.hh" +#include "chart/v3/titers.hh" +#include "chart/v3/serum-circles.hh" + +// ====================================================================== +// P2 milestone A — headless native render of a chart's on-chart *semantic* styling +// (c["R"] named styles + c["p"] base plot-spec), mirroring kateri's set_style + get_pdf +// for the report by-clade maps. Implements the §1.2 resolver and the §2.2 viewport +// transform+recenter+viewport chain in C++, then draws via cc/draw/cairo-surface. +// +// Viewport chain (the key fidelity step, derived from kateri lib/src/viewport.dart +// roundAndRecenter + lib/src/socket-events.dart get_viewport, and confirmed by +// py/ae/report/chart_modifier.py::export_mapi_for_signature_pages): +// * transformed layout = projection.transformed_layout() (stored 2x2 transform applied, +// NO Y-flip — post-transform Y is already screen-down). +// * kateri computes a native viewport = axis-aligned hull of the transformed layout, +// then roundAndRecenter: roundedSize = ceil(hull_extent + 1) per axis, and it shifts +// every point by adjust = roundedSize/2 - hull_centre so the native frame becomes +// [0,0,roundedSize]. The style's stored viewport V=[x,y,w,h] (the "-reset" family) is +// the "used" viewport expressed in this recentered space. +// * Since we draw in the (non-recentered) transformed space directly, the equivalent +// absolute viewport is origin = V.origin - adjust = -roundedSize/2 + V.origin + +// hull_centre, size = V.size. This equals the export_mapi_for_signature_pages +// recovery (-native/2 + used + native_center). +// +// Point sizes / outline widths / font sizes are absolute device pixels (kateri "sizePixels", +// independent of canvas width: sizePixels * pixelSize * canvasScale == sizePixels). Base +// plot-spec sizes carry kateri's x5 (_sizeScale); semantic-modifier `s`/`o` are used as-is. +// ====================================================================== + +namespace ae::map_draw +{ + using namespace ae::chart::v3; + + namespace + { + // ---- colour model (kateri lib/src/color.dart ColorAndModifier) ---- + // A style colour is a base colour string plus an optional deferred modifier. The + // report uses two modifiers: ":pale" (desaturate + lighten) and ":bright" (clear the + // pale modifier, keeping the base colour). Milestone E tracks the (base, pale) pair + // per point exactly like kateri, so a ":pale" background (e.g. -pale on the serology + // map) really pales the clade colours and a subsequent ":bright" (vaccine/serology + // highlight) brings them back to full colour. + enum class ColorAction { keep, set, pale, bright }; + struct ParsedColor + { + ColorAction action{ColorAction::keep}; + ::Color color{TRANSPARENT}; // valid only when action == set + }; + + ParsedColor parse_color(const ae::draw::v2::Color& c) + { + if (c.empty()) + return {}; + const std::string& s = c.blocks()[0]; + if (s.empty()) + return {}; + if (s[0] == ':') { + if (s == ":pale") + return {ColorAction::pale, {}}; + if (s == ":bright") + return {ColorAction::bright, {}}; + return {}; // unknown modifier: keep current colour (kateri warns + ignores) + } + if (s == "T" || s == "transparent") + return {ColorAction::set, TRANSPARENT}; + return {ColorAction::set, ::Color{std::string_view{s}}}; + } + + // kateri ColorAndModifier.color for the ":pale" modifier (pale factor 0.4). + ::Color pale_color(::Color base) + { + constexpr double pale_factor = 0.4; + acmacs::color::HSV hsv{base}; + if (hsv.s > 0.0) { + hsv.s = hsv.s * pale_factor; + hsv.v = hsv.v + (1.0 - hsv.v) * (1.0 - pale_factor); + } + else { // grey / black / white (saturation 0): lighten value only + double val = hsv.v / pale_factor; + if (val > 1.0) + val = 1.0; + else if (hsv.v == 0.0) + val = 0.5; // black -> mid grey + hsv.v = val; + } + const uint32_t alpha = base.raw_value() & 0xFF000000u; // preserve transparency byte + return ::Color{alpha | (hsv.rgb() & 0x00FFFFFFu)}; + } + + // Apply a parsed colour to a (base, pale) pair (kateri ColorAndModifier.modify). + void apply_color(const ParsedColor& pc, ::Color& color, bool& pale) + { + switch (pc.action) { + case ColorAction::keep: + break; + case ColorAction::set: + color = pc.color; + pale = false; + break; + case ColorAction::bright: + pale = false; // clear pale modifier, base colour unchanged + break; + case ColorAction::pale: + pale = true; + break; + } + } + + // Parse a colour held as a plain std::string (the serum-circle / serum-coverage fow + // fields carry std::string, not ae::draw::v2::Color). Mirrors parse_color(). + ParsedColor parse_color_str(std::string_view s) + { + if (s.empty()) + return {}; + if (s[0] == ':') { + if (s == ":pale") + return {ColorAction::pale, {}}; + if (s == ":bright") + return {ColorAction::bright, {}}; + return {}; + } + if (s == "T" || s == "transparent") + return {ColorAction::set, TRANSPARENT}; + return {ColorAction::set, ::Color{s}}; + } + + // Concrete colour from a plain std::string (serum-circle outline/fill): a deferred + // modifier / empty string falls back to the given default. + ::Color concrete_color_str(std::string_view s, ::Color fallback) + { + const ParsedColor pc = parse_color_str(s); + return pc.action == ColorAction::set ? pc.color : fallback; + } + + // ---- dynamic::value helpers (selectors + semantic attribute matching) ---- + bool value_is_bool(const ae::dynamic::value& v) { return std::holds_alternative(v.data()); } + bool value_as_bool(const ae::dynamic::value& v) { return std::get(v.data()); } + + long value_as_long(const ae::dynamic::value& v) + { + return std::visit( + [](const T& c) -> long { + if constexpr (std::is_same_v) + return c; + else if constexpr (std::is_same_v) + return static_cast(c); + else + return -1; + }, + v.data()); + } + + double value_as_double(const ae::dynamic::value& v) + { + return std::visit( + [](const T& c) -> double { + if constexpr (std::is_same_v) + return c; + else if constexpr (std::is_same_v) + return static_cast(c); + else + return std::numeric_limits::quiet_NaN(); + }, + v.data()); + } + + // kateri Chart.homologousTiterForSerum (chart.dart): rank antigens sharing the serum's + // name by annotation / reassortant / passage similarity, return the best (lowest-rank) + // non-dont-care titer. Used for the serum-coverage fold threshold. + Titer homologous_titer_for_serum(const Chart& chart, serum_index serum_no) + { + const auto& serum = chart.sera()[serum_no]; + const auto& antigens = chart.antigens(); + const auto& titers = chart.titers(); + const ae::dynamic::value& serum_p = serum.semantic().get("p"); + int best_rank = 1024; + Titer best_titer{"*"}; + for (const auto ag_no : antigens.size()) { + const auto& antigen = antigens[ag_no]; + if (serum.name() != antigen.name()) + continue; + int rank = 0; + if (serum.annotations() != antigen.annotations()) + rank += 16; + if (serum.reassortant() != antigen.reassortant()) + rank += 8; + if (serum.passage() != antigen.passage()) { + rank += 4; + if (!(serum_p == antigen.semantic().get("p"))) + rank += 2; + } + const Titer titer = titers.titer(ag_no, serum_no); + if (!titer.is_dont_care() && rank < best_rank) { + best_rank = rank; + best_titer = titer; + } + } + return best_titer; + } + + // A serum circle collected during modifier resolution, drawn (delayed) on top of the + // points — kateri DrawOn.delayedSerumCircle / SerumCircle.draw. Geometry is kept in + // world/map units (radius_world) so it can be mapped to device once the viewport is fixed. + struct SerumCircleR + { + size_t point{0}; // point index (serum) whose coords centre the circle + double radius_world{0.0}; // radius in map units (== serum-circle semantic radius) + ::Color outline{BLUE}; + ::Color fill{TRANSPARENT}; + double outline_width{1.0}; + long dash{0}; + bool has_angles{false}; + double angle0{0.0}, angle1{0.0}; + ::Color radius_outline{BLACK}; + double radius_outline_width{1.0}; + long radius_dash{0}; + }; + + // A resolved per-point render state. + struct PR + { + ::Color fill{TRANSPARENT}; + bool fill_pale{false}; + ::Color outline{BLACK}; + bool outline_pale{false}; + double outline_width{1.0}; + double size{10.0}; // device px (diameter / box side) + point_shape::Shape shape{point_shape::Circle}; + bool shown{true}; + bool has_label{false}; + double label_dx{0.0}, label_dy{1.0}; + double label_size{20.0}; + std::string label_text{}; + }; + + struct LegendRowR + { + int priority{0}; + std::string text{}; + ::Color fill{TRANSPARENT}; + bool fill_pale{false}; // kateri swatch tracks the (base, pale) pair like a point + ::Color outline{BLACK}; + bool outline_pale{false}; + double outline_width{1.0}; + point_shape::Shape shape{point_shape::Circle}; + size_t count{0}; + }; + + // Resolution accumulator (§1.2): flat modifier list, viewport, merged legend, title. + struct Resolved + { + std::vector modifiers{}; + std::optional viewport{}; + semantic::Legend legend{}; + semantic::Title title{}; + bool title_set{false}; + }; + + void merge_legend(semantic::Legend& into, const semantic::Legend& from) + { + if (from.shown.has_value()) into.shown = from.shown; + if (from.add_counter.has_value()) into.add_counter = from.add_counter; + if (from.point_size.has_value()) into.point_size = from.point_size; + if (from.show_rows_with_zero_count.has_value()) into.show_rows_with_zero_count = from.show_rows_with_zero_count; + if (from.box.has_value()) into.box = from.box; + if (from.row_style.has_value()) into.row_style = from.row_style; + if (from.title.has_value()) into.title = from.title; + } + + // Walk a front style, recursively expanding {R:} references into a flat modifier + // list (§1.2). Undefined references are tolerated. Viewport is last-writer-wins across + // the traversal; legend fields merge; title comes from the front (depth 0) style. + void resolve(const semantic::Styles& styles, const std::string& name, Resolved& out, int depth) + { + const semantic::Style* st = styles.find_if_exists(name); + if (st == nullptr) + return; // tolerate undefined ref (e.g. "-new-1" on a chart with no previous) + if (st->viewport.has_value()) + out.viewport = st->viewport; + merge_legend(out.legend, st->legend); + if (depth == 0) { + out.title = st->plot_title; + out.title_set = true; + } + for (const auto& m : st->modifiers) { + if (!m.parent.empty()) + resolve(styles, m.parent, out, depth + 1); + else + out.modifiers.push_back(&m); + } + } + + // kateri Antigen.withinDateRange (chart.dart): a "!D": [first, last] range test. + bool within_date_range(std::string_view date, std::string_view first, std::string_view last) + { + if (date.empty()) + return first.empty(); // dateless antigen is "at the beginning of all dates" + return (first.empty() || first <= date) && (last.empty() || last > date); + } + + // kateri semanticMatch (chart.dart): one selector key/value vs the point's attributes. + // * a bool selector value is a *presence* test (attr present == value); + // * otherwise a missing attribute never matches; + // * a list-valued attribute (clades "C") matches if it contains the value; + // * otherwise scalar equality. + bool semantic_match(std::string_view key, const ae::dynamic::value& selval, const SemanticAttributes& sem) + { + const ae::dynamic::value& attr = sem.get(key); + if (value_is_bool(selval)) + return value_as_bool(selval) == !attr.is_null(); + if (attr.is_null()) + return false; + if (const auto* arr = std::get_if(&attr.data())) { // clades list: contains + for (const auto& el : arr->data()) { + if (el == selval) + return true; + } + return false; + } + return attr == selval; + } + + // Match a whole selector object against one point (kateri PlotSpec.selectPoints::match): + // handles "!i" (per-side index), "!D" (antigen date range), and semantic keys. An empty + // (or non-object) selector matches every candidate. + bool point_matches(const ae::dynamic::value& selector, const SemanticAttributes& sem, long agsr_no, std::string_view date, bool is_antigen) + { + const auto* obj = std::get_if(&selector.data()); + if (obj == nullptr) + return true; + for (const auto& [key, val] : obj->data()) { + if (key == "!i") { + if (agsr_no != value_as_long(val)) + return false; + } + else if (key == "!D") { + if (!is_antigen) + return false; + std::string_view first{}, last{}; + if (const auto* arr = std::get_if(&val.data())) { + const auto& d = arr->data(); + if (d.size() >= 1) + first = d[0].as_string_or_empty(); + if (d.size() >= 2) + last = d[1].as_string_or_empty(); + } + if (!within_date_range(date, first, last)) + return false; + } + else if (!semantic_match(key, val, sem)) + return false; + } + return true; + } + + } // namespace + + // ---------------------------------------------------------------------- + + void export_styled_map(const Chart& chart, projection_index projection_no, std::string_view style_name, double width, const std::filesystem::path& output) + { + if (chart.projections().empty()) + throw std::runtime_error{"cannot draw styled map: chart has no projections"}; + if (projection_no >= chart.projections().size()) + throw std::runtime_error{"cannot draw styled map: projection index out of range"}; + + const auto& projection = chart.projections()[projection_no]; + const Layout layout = projection.transformed_layout(); + + const size_t n_antigens = chart.antigens().size().get(); + const size_t n_sera = chart.sera().size().get(); + const size_t n_points = layout.number_of_points().get(); + + // ---- semantic accessor per point ---- + const auto sem_of = [&](size_t i) -> const SemanticAttributes& { + if (i < n_antigens) + return chart.antigens()[antigen_index{i}].semantic(); + return chart.sera()[serum_index{i - n_antigens}].semantic(); + }; + + // ---- default per-point render state (kateri PlotSpecSemantic.activate) ---- + // The report's by-clade maps use kateri's *semantic* plot spec, which is seeded from a + // fresh grey DEFAULT (initDefaultPointSpecs + makeDefaultPointSpecs, plot_spec.dart) and + // then has the c["R"] modifiers applied — it deliberately IGNORES the c["p"] legacy plot + // spec. Seeding from c["p"] instead (as this did) leaks leftover base clade colours onto + // the non-highlighted points: on a chart where only a small subset matches a clade + // selector (e.g. H3 clades-v1) the *whole* cloud shows base colour instead of greying. + // kateri's default (semantic activate, testAntigenFill=gray80, outline=gray80): + // * test antigen : fill gray80, outline gray80, size testSize(20), shape circle/egg + // * reference ag : fill transparent, outline gray80, size refSize(32), shape circle/egg + // * serum : fill transparent, outline gray80, size refSize(32), shape box/uglyegg + // (egg/reassortant → egg (antigen) / uglyegg (serum); reassortant rotation not modelled) + const antigen_indexes reference = chart.reference(); + const ::Color gray80{0xCCCCCC}; + constexpr double kTestSize = 20.0, kRefSize = 32.0; // kateri PointPlotSpec.testSize/refSize + // kateri _AntigenSerum.isEgg: prefer the semantic "p" attribute ("e"=egg), else passage. + const auto is_egg = [](const auto& agsr) -> bool { + const ae::dynamic::value& p = agsr.semantic().get("p"); + if (!p.is_null()) + return p.as_string_or_empty() == std::string_view{"e"}; + return agsr.passage().is_egg(); + }; + std::vector pr(n_points); + for (size_t i = 0; i < n_antigens; ++i) { + PR& p = pr[i]; + const auto& ag = chart.antigens()[antigen_index{i}]; + const bool is_ref = reference.contains(antigen_index{i}); + const bool egg_or_reass = is_egg(ag) || !ag.reassortant().empty(); + p.outline = gray80; + p.outline_width = 1.0; + p.fill = is_ref ? TRANSPARENT : gray80; + p.size = is_ref ? kRefSize : kTestSize; + p.shape = egg_or_reass ? point_shape::Egg : point_shape::Circle; + } + for (size_t i = n_antigens; i < n_points; ++i) { + PR& p = pr[i]; + const auto& sr = chart.sera()[serum_index{i - n_antigens}]; + const bool egg_or_reass = is_egg(sr) || !sr.reassortant().empty(); + p.outline = gray80; + p.outline_width = 1.0; + p.fill = TRANSPARENT; + p.size = kRefSize; + p.shape = egg_or_reass ? point_shape::UglyEgg : point_shape::Box; + } + + // ---- default draw order (kateri makeDefaultDrawingOrder): sera, then reference + // antigens, then test antigens — again the semantic default, not c["p"]["d"]. ---- + std::vector order; + for (size_t i = n_antigens; i < n_points; ++i) + order.push_back(i); // sera (bottom) + for (const auto ag : reference) + order.push_back(ag.get()); // reference antigens + for (size_t i = 0; i < n_antigens; ++i) { + if (!reference.contains(antigen_index{i})) + order.push_back(i); // test antigens (top) + } + + // ---- resolve the front style (§1.2) ---- + Resolved resolved; + resolve(chart.styles(), std::string{style_name}, resolved, 0); + + std::vector legend_rows; + std::vector serum_circle_list; // collected (milestone F), drawn on top of points + + // helper: raise the given points to the top of the draw order (kateri raiseLowerPoints "r") + const auto raise_points = [&](const std::vector& to_raise) { + if (to_raise.empty()) + return; + std::vector is_raised(n_points, false); + for (const size_t i : to_raise) + if (i < n_points) + is_raised[i] = true; + std::vector kept, moved; + for (const size_t i : order) { + if (is_raised[i]) + moved.push_back(i); + else + kept.push_back(i); + } + order.clear(); + order.insert(order.end(), kept.begin(), kept.end()); + order.insert(order.end(), moved.begin(), moved.end()); + }; + + // ---- apply modifiers in flat order ---- + for (const semantic::StyleModifier* mp : resolved.modifiers) { + const semantic::StyleModifier& m = *mp; + // candidate range by antigens/sera select + size_t cand_begin = 0, cand_end = n_points; + if (m.select_antigens_sera == semantic::SelectAntigensSera::antigens_only) { + cand_begin = 0; + cand_end = n_antigens; + } + else if (m.select_antigens_sera == semantic::SelectAntigensSera::sera_only) { + cand_begin = n_antigens; + cand_end = n_points; + } + + std::vector matched; + for (size_t i = cand_begin; i < cand_end; ++i) { + const bool is_ag = i < n_antigens; + const long agsr = is_ag ? static_cast(i) : static_cast(i - n_antigens); + const std::string_view date = is_ag ? static_cast(chart.antigens()[antigen_index{i}].date()) : std::string_view{}; + if (point_matches(m.selector, sem_of(i), agsr, date, is_ag)) + matched.push_back(i); + } + + // apply point-style fields (colours track the kateri (base, pale) pair) + const PointStyle& ms = m.point_style; + for (const size_t i : matched) { + PR& p = pr[i]; + apply_color(parse_color(ms.fill()), p.fill, p.fill_pale); + apply_color(parse_color(ms.outline()), p.outline, p.outline_pale); + if (ms.outline_width().has_value()) + p.outline_width = *ms.outline_width(); + if (ms.size().has_value()) + p.size = *ms.size(); // semantic-modifier size is absolute px (no x5) + if (ms.shape().has_value()) + p.shape = ms.shape()->get(); + if (ms.shown().has_value()) + p.shown = *ms.shown(); + if (ms.label().text.has_value() && !ms.label().text->empty()) { + p.has_label = true; + p.label_text = *ms.label().text; + p.label_dx = ms.label().offset.x; + p.label_dy = ms.label().offset.y; + p.label_size = ms.label().size; + } + } + + // drawing order raise / lower + if (m.order != semantic::DrawingOrderModifier::no_change && !matched.empty()) { + std::vector is_matched(n_points, false); + for (const size_t i : matched) + if (i < n_points) + is_matched[i] = true; + std::vector kept, moved; + for (const size_t i : order) { + if (is_matched[i]) + moved.push_back(i); + else + kept.push_back(i); + } + order.clear(); + if (m.order == semantic::DrawingOrderModifier::raise) { // matched go on top (drawn last) + order.insert(order.end(), kept.begin(), kept.end()); + order.insert(order.end(), moved.begin(), moved.end()); + } + else { // lower: matched go to the bottom (drawn first) + order.insert(order.end(), moved.begin(), moved.end()); + order.insert(order.end(), kept.begin(), kept.end()); + } + } + + // legend row (kateri applyEntry): the swatch copies the first matched point's + // resolved spec (fill/outline + their :pale state, outline width), shape forced + // circle; with no matched point it applies this modifier's F/O to a fresh default. + if (!m.legend.text.empty() || m.legend.priority != 0) { + LegendRowR row; + row.priority = m.legend.priority; + row.text = m.legend.text; + if (!matched.empty()) { + const PR& mp0 = pr[matched[0]]; + row.fill = mp0.fill; + row.fill_pale = mp0.fill_pale; + row.outline = mp0.outline; + row.outline_pale = mp0.outline_pale; + row.outline_width = mp0.outline_width; + } + else { // fresh default spec + this modifier's F/O (kateri modifyPointPlotSpec(entry, PointPlotSpec())) + row.fill = TRANSPARENT; + row.outline = gray80; + apply_color(parse_color(ms.fill()), row.fill, row.fill_pale); + apply_color(parse_color(ms.outline()), row.outline, row.outline_pale); + if (ms.outline_width().has_value()) + row.outline_width = *ms.outline_width(); + } + if (ms.shape().has_value()) + row.shape = ms.shape()->get(); + row.count = matched.size(); + legend_rows.push_back(std::move(row)); + } + // kateri's "make legend for pale clades pale too" hack (plot_spec.dart applyEntry): + // a global colour modifier (no legend row, no selector) whose fill or outline is a + // deferred :pale/:bright re-applies both F and O to every existing legend swatch, so + // the -pale on the serology map pales the -clades swatches to match the paled points. + else { + const bool no_selector = std::get_if(&m.selector.data()) == nullptr; + const ParsedColor pf = parse_color(ms.fill()); + const ParsedColor po = parse_color(ms.outline()); + const bool deferred = pf.action == ColorAction::pale || pf.action == ColorAction::bright + || po.action == ColorAction::pale || po.action == ColorAction::bright; + if (no_selector && deferred) { + for (auto& row : legend_rows) { + apply_color(pf, row.fill, row.fill_pale); + apply_color(po, row.outline, row.outline_pale); + } + } + } + + // ---- serum circle (milestone F): collect for delayed drawing ---- + // kateri PlotSpec.serumCircleData + DrawOn.delayedSerumCircle. The radius comes from + // the serum's CI{fold} semantic attribute (e=empirical, t=theoretical); a missing + // radius falls back to `fold` map units drawn heavily dashed (kateri dash=100). + if (m.serum_circle.has_value()) { + const semantic::serum_circle_style_t& sc = *m.serum_circle; + const std::string ci_key = fmt::format("CI{}", static_cast(sc.fold)); + for (const size_t i : matched) { + if (i < n_antigens) + continue; // sera only + const auto c = layout[point_index{i}]; + if (!c.exists()) + continue; + const serum_index sr{i - n_antigens}; + std::optional radius_world; + const ae::dynamic::value& ci = chart.sera()[sr].semantic().get(ci_key); + if (ci.is_object()) { + const ae::dynamic::value& r = ci[sc.theoretical ? std::string_view{"t"} : std::string_view{"e"}]; + if (!r.is_null()) { + if (const double rv = value_as_double(r); !std::isnan(rv)) + radius_world = rv; + } + } + long dash = sc.dash; + if (!radius_world.has_value()) { + if (!sc.fallback) + continue; + radius_world = sc.fold; // fallback: draw a circle of `fold` map units, heavily dashed + dash = 100; + } + SerumCircleR out; + out.point = i; + out.radius_world = *radius_world; + out.outline = concrete_color_str(sc.outline, BLUE); + out.fill = concrete_color_str(sc.fill, TRANSPARENT); + out.outline_width = sc.outline_width; + out.dash = dash; + if (sc.angles.has_value()) { + out.has_angles = true; + out.angle0 = sc.angles->first; + out.angle1 = sc.angles->second; + out.radius_outline = sc.radius_outline.has_value() ? concrete_color_str(*sc.radius_outline, out.outline) : out.outline; + out.radius_outline_width = sc.radius_outline_width.value_or(out.outline_width); + out.radius_dash = sc.radius_dash.value_or(out.dash); + } + serum_circle_list.push_back(out); + } + } + + // ---- serum coverage (milestone F): restyle antigens within/outside the fold ---- + // kateri PlotSpec.applySerumCoverage: split the serum's antigens by whether their + // titre is within `fold` of the homologous titre, recolour each group's fill/outline, + // and raise them (within first, then outside → outside on top). + if (m.serum_coverage.has_value()) { + const semantic::serum_coverage_style_t& cov_spec = *m.serum_coverage; + for (const size_t i : matched) { + if (i < n_antigens) + continue; // sera only + const serum_index sr{i - n_antigens}; + serum_coverage_serum_t cov; + try { + cov = serum_coverage(chart.titers(), homologous_titer_for_serum(chart, sr), sr, serum_circle_fold{cov_spec.fold}); + } + catch (const std::exception&) { + continue; // kateri warns and skips (no homologous titre / titre too low) + } + const auto restyle = [&](const antigen_indexes& idxs, const semantic::point_style_fow_t& fow, std::vector& raised) { + for (const auto ag : idxs) { + const size_t pi = ag.get(); + PR& p = pr[pi]; + apply_color(parse_color_str(fow.fill), p.fill, p.fill_pale); + apply_color(parse_color_str(fow.outline), p.outline, p.outline_pale); + p.outline_width = fow.outline_width; + raised.push_back(pi); + } + }; + std::vector within_pts, outside_pts; + restyle(cov.within, cov_spec.within, within_pts); + restyle(cov.outside, cov_spec.outside, outside_pts); + raise_points(within_pts); + raise_points(outside_pts); + } + } + } + + // ---- viewport chain (§2.2) ---- + constexpr double inf = std::numeric_limits::infinity(); + double min_x{inf}, min_y{inf}, max_x{-inf}, max_y{-inf}; + for (const auto pn : layout.number_of_points()) { + if (const auto c = layout[pn]; c.exists()) { + min_x = std::min(min_x, c[DIMX]); + min_y = std::min(min_y, c[DIMY]); + max_x = std::max(max_x, c[DIMX]); + max_y = std::max(max_y, c[DIMY]); + } + } + if (min_x > max_x) + throw std::runtime_error{"cannot draw styled map: no points with coordinates"}; + const double centre_x = (min_x + max_x) / 2.0; + const double centre_y = (min_y + max_y) / 2.0; + const double rounded_x = std::ceil(max_x - min_x + 1.0); + const double rounded_y = std::ceil(max_y - min_y + 1.0); + + double vp_x, vp_y, vp_w, vp_h; + if (resolved.viewport.has_value()) { + const auto& V = *resolved.viewport; + vp_w = V.width; + vp_h = V.height; + vp_x = -rounded_x / 2.0 + static_cast(V.x) + centre_x; + vp_y = -rounded_y / 2.0 + static_cast(V.y) + centre_y; + } + else { // no style viewport => kateri's native (recentered) frame + vp_w = rounded_x; + vp_h = rounded_y; + vp_x = centre_x - rounded_x / 2.0; + vp_y = centre_y - rounded_y / 2.0; + } + + const double image_w = width; + const double image_h = width * vp_h / vp_w; + const auto dev_x = [=](double x) { return (x - vp_x) / vp_w * image_w; }; + const auto dev_y = [=](double y) { return (y - vp_y) / vp_h * image_h; }; // NO Y-flip + + ae::draw::CairoPdf surface{output, image_w, image_h}; + surface.background(WHITE); + + // ---- grid (kateri grid: colour #CCCCCC as rendered in the golden, 1px, step 1 map unit + // from vp origin; draw_on.dart's abstract default 0xCCCCCC, matched to the golden) ---- + { + const ::Color grid{0xCCCCCC}; + const double step_x = image_w / vp_w; + const double step_y = image_h / vp_h; + for (double gx = 0.0; gx <= image_w + 0.5; gx += step_x) + surface.line(gx, 0.0, gx, image_h, grid, 1.0); + for (double gy = 0.0; gy <= image_h + 0.5; gy += step_y) + surface.line(0.0, gy, image_w, gy, grid, 1.0); + } + + // ---- points (draw order: first = bottom) ---- + const auto draw_point = [&](size_t i) { + const auto c = layout[point_index{i}]; + if (!c.exists() || !pr[i].shown) + return; + const PR& p = pr[i]; + const double cx = dev_x(c[DIMX]), cy = dev_y(c[DIMY]); + const double s = p.size; + const double ow = p.outline_width; + const ::Color fill = p.fill_pale ? pale_color(p.fill) : p.fill; // deferred ":pale" (kateri) + const ::Color outline = p.outline_pale ? pale_color(p.outline) : p.outline; + switch (p.shape) { + case point_shape::Box: + surface.square(cx, cy, s, outline, ow, fill); + break; + case point_shape::Triangle: + surface.triangle(cx, cy, s / 2.0, outline, ow, fill); + break; + case point_shape::Egg: + case point_shape::UglyEgg: + surface.egg(cx, cy, s, outline, ow, fill); + break; + case point_shape::Circle: + default: + surface.circle(cx, cy, s / 2.0, outline, ow, fill); + break; + } + }; + for (const size_t i : order) + draw_point(i); + + // ---- serum circles (milestone F): drawn on top of the points (kateri drawDelayed) ---- + // The stored radius is in map units; convert to device with the (uniform) viewport scale. + { + const double scale = image_w / vp_w; // == image_h / vp_h (uniform) + for (const SerumCircleR& s : serum_circle_list) { + const auto c = layout[point_index{s.point}]; + if (!c.exists()) + continue; + const double cx = dev_x(c[DIMX]), cy = dev_y(c[DIMY]); + const double r = s.radius_world * scale; + constexpr double two_pi = 2.0 * std::numbers::pi; + // radius-line endpoint at a given angle (clockwise from 12 o'clock; matches arc()). + const auto endpoint = [&](double angle) { return std::pair{cx + std::sin(angle) * r, cy - std::cos(angle) * r}; }; + + if (!s.has_angles) { // whole circle + if (s.dash == 0) { + surface.circle(cx, cy, r, s.outline, s.outline_width, s.fill); + } + else { // kateri circleDashed -> sectorDashed(wholeCircle): fill then dashed arcs + if (!s.fill.is_transparent()) + surface.circle(cx, cy, r, TRANSPARENT, 0.0, s.fill); + const double gap = std::numbers::pi / static_cast(s.dash) / 2.0; + const double single = two_pi / static_cast(s.dash); + for (long i = 0; i < s.dash; ++i) { + const double a0 = gap + single * static_cast(i); + surface.arc(cx, cy, r, a0, a0 + single - gap * 2.0, s.outline, s.outline_width); + } + } + } + else { // angular sector with radius lines (kateri sector / sectorDashed) + if (!s.fill.is_transparent()) + surface.sector(cx, cy, r, s.angle0, s.angle1, TRANSPARENT, 0.0, s.fill); + if (s.dash == 0) { + surface.arc(cx, cy, r, s.angle0, s.angle1, s.outline, s.outline_width); + } + else { + const double gap = std::numbers::pi / static_cast(s.dash) / 2.0; + const double single = two_pi / static_cast(s.dash); + const double span = std::abs(s.angle1 - s.angle0); + const long dashes = std::lround(span / single); + for (long i = 0; i < dashes; ++i) { + const double a0 = s.angle0 + gap + single * static_cast(i); + surface.arc(cx, cy, r, a0, a0 + single - gap * 2.0, s.outline, s.outline_width); + } + } + if (s.radius_outline_width > 0.0 && !s.radius_outline.is_transparent()) { + const auto [e0x, e0y] = endpoint(s.angle0); + const auto [e1x, e1y] = endpoint(s.angle1); + surface.line(cx, cy, e0x, e0y, s.radius_outline, s.radius_outline_width); + surface.line(cx, cy, e1x, e1y, s.radius_outline, s.radius_outline_width); + } + } + } + } + + // ---- point labels (on top of points) ---- + for (const size_t i : order) { + if (!pr[i].has_label) + continue; + const auto c = layout[point_index{i}]; + if (!c.exists()) + continue; + const PR& p = pr[i]; + const double cx = dev_x(c[DIMX]), cy = dev_y(c[DIMY]); + const auto [tw, th] = surface.text_size(p.label_text, p.label_size, true); + const double point_r = (p.size + p.outline_width) / 2.0; // device px (kateri pointSize) + // kateri addPointLabel::labelOffset (draw_on.dart), in device px; baseline-left anchor. + const auto lab_off = [point_r](double off, double extent, bool vertical) -> double { + if (off >= 1.0) + return point_r * off + (vertical ? extent : 0.0); + if (off > -1.0) + return point_r * off + (vertical ? extent * (off + 1.0) / 2.0 : extent * (off - 1.0) / 2.0); + return point_r * off - (vertical ? 0.0 : extent); + }; + const double lx = cx + lab_off(p.label_dx, tw, false); + const double ly = cy + lab_off(p.label_dy, th, true); + // kateri gives every point label a default thin white halo (pointLabelHaloWidthFactor) + // so it reads over the dark point cloud; without it the black glyphs vanish into the + // points they sit on. Positioning is data-driven (the style's per-label offset) exactly + // as kateri does — kateri performs no collision auto-placement, so neither do we. + constexpr double kPointLabelHaloWidthFactor = 0.04; + surface.text_font(lx, ly, p.label_text, p.label_size, BLACK, false, false, p.label_size * kPointLabelHaloWidthFactor); + } + + // ---- legend (kateri _Defaults.legend: bottom-left "Bl", offset (10,-10), white box, + // black 1px border, padding v5/h10; point size + row text/interline from -clades) ---- + const bool legend_shown = resolved.legend.shown.value_or(true) && !legend_rows.empty(); + if (legend_shown) { + std::sort(legend_rows.begin(), legend_rows.end(), [](const LegendRowR& a, const LegendRowR& b) { return a.priority < b.priority; }); + const bool add_counter = resolved.legend.add_counter.value_or(false); + const double point_size = resolved.legend.point_size.value_or(20.0); + double text_size = 20.0, interline = 0.3; + if (resolved.legend.row_style.has_value()) { + if (resolved.legend.row_style->font_size.has_value()) + text_size = *resolved.legend.row_style->font_size; + if (resolved.legend.row_style->interline.has_value()) + interline = *resolved.legend.row_style->interline; + } + const double pad_l = 10.0, pad_r = 10.0, pad_t = 5.0, pad_b = 5.0; + const double point_space = point_size * (interline + 1.2); + const double count_left_pad = add_counter ? text_size * 1.0 : 0.0; + + double max_text_w = 0.0, row_h = 0.0, max_count_w = 0.0; + for (const auto& r : legend_rows) { + const auto [w, h] = surface.text_size(r.text, text_size, true); + max_text_w = std::max(max_text_w, w); + row_h = std::max(row_h, h); + if (add_counter) { + const auto [cw, ch] = surface.text_size(fmt::format("{}", r.count), text_size, true); + (void)ch; + max_count_w = std::max(max_count_w, cw); + } + } + const double n = static_cast(legend_rows.size()); + const double box_w = max_text_w + point_space + count_left_pad + max_count_w + pad_l + pad_r; + const double box_h = row_h + row_h * (n - 1.0) * (interline + 1.0) + row_h * 0.4 + pad_t + pad_b; + const double box_x = 0.0 + 10.0; // vp.left device (=0) + offset.dx (10 px) + const double box_y = image_h - 10.0 - box_h; // vp.bottom device (=image_h) + offset.dy(-10) - height + surface.rectangle(box_x, box_y, box_w, box_h, BLACK, 1.0, WHITE); + + const double dx = box_x + pad_l; + double baseline = box_y + pad_t + row_h; // kateri: box.origin.dy + textSize[0].height + padding.top + for (const auto& r : legend_rows) { + const ::Color rfill = r.fill_pale ? pale_color(r.fill) : r.fill; + const ::Color routline = r.outline_pale ? pale_color(r.outline) : r.outline; + surface.circle(dx + point_size / 2.0, baseline - row_h * 0.35, point_size / 2.0, routline, r.outline_width, rfill); + surface.text_font(dx + point_space, baseline, r.text, text_size, BLACK, false, false); + if (add_counter) { + const auto [cw, ch] = surface.text_size(fmt::format("{}", r.count), text_size, true); + (void)ch; + surface.text_font(dx + point_space + max_text_w + count_left_pad + max_count_w - cw, baseline, fmt::format("{}", r.count), text_size, BLACK, false, false); + } + baseline += row_h * (interline + 1.0); + } + } + + // ---- title (kateri _Defaults.title "tl", offset from box O; helvetica bold/normal) ---- + // The "info-" front styles carry a whitespace-only title (" ") — a deliberate blank + // title (§1.2); skip drawing it entirely so nothing is rendered. + const auto title_is_blank = [](std::string_view s) { return s.find_first_not_of(" \t") == std::string_view::npos; }; + if (resolved.title_set && resolved.title.shown.value_or(true) && resolved.title.text.text.has_value() && !title_is_blank(*resolved.title.text.text)) { + const auto& t = resolved.title; + double off_x = 30.0, off_y = 30.0; // kateri title default offset + if (t.box.has_value() && t.box->offset.has_value()) { + off_x = (*t.box->offset)[0]; + off_y = (*t.box->offset)[1]; + } + const double font = t.text.font_size.value_or(28.0); + const bool bold = t.text.font_weight.value_or("normal") == "bold"; + const bool italic = t.text.font_slant.value_or("normal") == "italic"; + ::Color col = BLACK; + if (t.text.color.has_value()) + col = ::Color{static_cast(*t.text.color)}; + // Multi-line title (kateri PlotText: the "t" string is split on newlines via + // LineSplitter, then paintTitle draws each line advancing the baseline by + // lineHeight*(interline+1)). origin "tl": box top-left at device (off_x, off_y); + // kateri draws the first baseline at box.origin.dy + textHeight + padding.top + // (padding 0). Baseline-left anchor. + const double interline = t.text.interline.value_or(0.2); + std::string_view rest{*t.text.text}; + double baseline = 0.0; + bool first = true; + while (true) { + const size_t nl = rest.find('\n'); + const std::string_view line = rest.substr(0, nl); + const auto [lw, lh] = surface.text_size(line, font, true); + (void)lw; + if (first) { + baseline = off_y + lh; + first = false; + } + surface.text_font(off_x, baseline, line, font, col, bold, italic); + baseline += lh * (interline + 1.0); + if (nl == std::string_view::npos) + break; + rest.remove_prefix(nl + 1); + } + } + } + +} // namespace ae::map_draw + +// ---------------------------------------------------------------------- diff --git a/cc/py/map-draw.cc b/cc/py/map-draw.cc index 99328c2..1d8da36 100644 --- a/cc/py/map-draw.cc +++ b/cc/py/map-draw.cc @@ -54,6 +54,17 @@ void ae::py::map_draw(pybind11::module_& mdl) }, "primary_ace"_a, "secondary_ace"_a, "output"_a, "size"_a = 800.0, "mapi"_a = std::nullopt, "coloring"_a = std::nullopt, pybind11::doc("Procrustes render (AD make_pc): draw primary with arrows to secondary (threshold 0.3), title 'RMS: x.xxxx'.")); + + // --- P2 milestone A: styled (semantic c["R"] + c["p"]) render, kateri-compatible drop-in --- + sub.def( + "export_styled_map", + [](const std::filesystem::path& ace, const std::filesystem::path& output, const std::string& style, double width, size_t projection_no) { + const Chart chart{ace}; + ae::map_draw::export_styled_map(chart, projection_index{projection_no}, style, width, output); + }, + "ace"_a, "output"_a, "style"_a, "width"_a = 800.0, "projection_no"_a = 0, + pybind11::doc("Render a chart's on-chart semantic style (c[\"R\"] named style + c[\"p\"] base plot-spec) to output " + "(.png raster, else PDF), mirroring kateri set_style + get_pdf. P2 milestone A.")); } // ---------------------------------------------------------------------- diff --git a/meson.build b/meson.build index 9d5223e..83fccf4 100644 --- a/meson.build +++ b/meson.build @@ -127,6 +127,7 @@ sources_py = [ 'cc/py/hidb.cc', # --- hidb (subsystem #2) --- 'cc/py/map-draw.cc', # --- map-draw (subsystem #1, revived) --- 'cc/map-draw/draw.cc', # --- map-draw (subsystem #1, revived) --- + 'cc/map-draw/styled-draw.cc', # --- map-draw P2 milestone A (styled/semantic render) --- 'cc/draw/cairo-surface.cc', # --- map-draw (subsystem #1, revived) --- ] @@ -318,7 +319,7 @@ geo_draw = executable( # --- map-draw (subsystem #1, revived) — headless antigenic-map renderer, fidelity # port of AD acmacs-map-draw ChartDraw for the chains-202105 web app. Reuses the # shared cc/draw/cairo-surface (Cairo linked here). See cc/map-draw/TODO.md. --- -sources_map_draw = ['cc/map-draw/draw.cc', 'cc/draw/cairo-surface.cc'] +sources_map_draw = ['cc/map-draw/draw.cc', 'cc/map-draw/styled-draw.cc', 'cc/draw/cairo-surface.cc'] map_draw = executable( 'map-draw', diff --git a/py/ae/report/chart_modifier.py b/py/ae/report/chart_modifier.py index 6c53dae..745d06e 100644 --- a/py/ae/report/chart_modifier.py +++ b/py/ae/report/chart_modifier.py @@ -522,14 +522,22 @@ def _clades_version_suffix(self, clade_style_name: str, infix: str): async def export_mapi_for_signature_pages(self, filename: Path, style: str): """Write a `.mapi` JSON file (map viewport plus vaccine markers) for the - signature-page renderer, querying kateri for the current viewport under `style`.""" - kateri.communicator.set_style(style) - viewport_data = await kateri.communicator.get_viewport() - viewport = [ - - viewport_data["native"][2] / 2.0 + viewport_data["used"][0] + viewport_data["native_center"][0], - - viewport_data["native"][3] / 2.0 + viewport_data["used"][1] + viewport_data["native_center"][1], - viewport_data["used"][2] - ] + signature-page renderer. The viewport comes from kateri (`set_style` + `get_viewport`) + by default; with `AE_REPORT_MAP_RENDERER=native` it is computed in-process from the + chart layout (kateri-free), see `map_renderer.sig_page_viewport`.""" + from . import map_renderer + if map_renderer.native_selected(): + # `style`'s own viewport is the report-set `self.viewport()` (the kateri "used" + # viewport); the native helper recentres it exactly as kateri's get_viewport does. + viewport = map_renderer.sig_page_viewport(self.chart, self.viewport(zoom_variant="")) + else: + kateri.communicator.set_style(style) + viewport_data = await kateri.communicator.get_viewport() + viewport = [ + - viewport_data["native"][2] / 2.0 + viewport_data["used"][0] + viewport_data["native_center"][0], + - viewport_data["native"][3] / 2.0 + viewport_data["used"][1] + viewport_data["native_center"][1], + viewport_data["used"][2] + ] fill_key = "fill" + self._clades_version_suffix(style, infix="_") data = { "loc:viewport": [{"N": "viewport", "abs": viewport}], diff --git a/py/ae/report/commander.py b/py/ae/report/commander.py index 03d6de5..03764f8 100644 --- a/py/ae/report/commander.py +++ b/py/ae/report/commander.py @@ -25,6 +25,8 @@ import ae.report.dirs from .main_loop import command, no_kateri, no_loop, headless from .chart_modifier import ChartModifier +from . import map_renderer +from .map_renderer import get_map_renderer # ====================================================================== @@ -114,11 +116,18 @@ async def export(self): chart_modifier = self.style() # do not await in parallel because current katteri protocol does not allow matching pdfs request and result for style_name in chart_modifier.export_styles(): - await self.export_pdf(style_name=style_name, output_filename=Path(".").resolve().joinpath(f"out.1.{style_name}.pdf")) + await self.export_pdf(style_name=style_name, output_filename=Path(".").resolve().joinpath(f"out.1.{style_name}.pdf"), chart=chart_modifier.chart) await self.export_mapi_for_signature_pages(chart_modifier=chart_modifier) - kateri.communicator.export_to_legacy(style=chart_modifier.style_for_legacy_plot_spec()) - chart = await kateri.communicator.get_chart() - chart.write(ae.report.dirs.VcmDirs.styled_filename()) + # Finalise styled.ace. With AE_REPORT_MAP_RENDERER=native the whole export runs + # kateri-free: serialise the in-memory styled chart directly (see + # map_renderer.write_styled_ace for why this is equivalent to the kateri bake). + # Unset/default keeps the exact previous kateri path. + if map_renderer.native_selected(): + map_renderer.write_styled_ace(chart_modifier.chart, ae.report.dirs.VcmDirs.styled_filename()) + else: + kateri.communicator.export_to_legacy(style=chart_modifier.style_for_legacy_plot_spec()) + chart = await kateri.communicator.get_chart() + chart.write(ae.report.dirs.VcmDirs.styled_filename()) @command @no_loop @@ -127,7 +136,7 @@ async def export_info(self): """`export_info` command: style the chart and export the info-map PDFs (headless).""" chart_modifier = self.style() for style_name in chart_modifier.export_info_styles(): - await self.export_pdf(style_name=style_name, output_filename=Path(".").resolve().joinpath(f"out.1.{style_name}.pdf")) + await self.export_pdf(style_name=style_name, output_filename=Path(".").resolve().joinpath(f"out.1.{style_name}.pdf"), chart=chart_modifier.chart) @command @no_loop @@ -167,7 +176,7 @@ async def serum_coverage_export(self, serum_selector: Callable | None = None, fo for et in ["e", "t"]: for zoom_variant in chart_modifier.zoom_variants(): style_name = f"sc-{serum_no:03d}-f{fold}-{et}{zoom_variant}" - await self.export_pdf(style_name=style_name, output_filename=self.serum_coverage_output_dir().joinpath(f"{style_name}.pdf")) + await self.export_pdf(style_name=style_name, output_filename=self.serum_coverage_output_dir().joinpath(f"{style_name}.pdf"), chart=chart_modifier.chart) self.serum_coverage_webpage(chart_modifier=chart_modifier) @command @@ -201,14 +210,13 @@ def download_from_previous(self, rotate: float | None = None): downloader.use_previous(ae.report.dirs.VcmDirs().find_previous_chart(), rotate=rotate).populate_from_seqdb().export_downloaded() return downloader - async def export_pdf(self, style_name: str, output_filename: Path): - """Request the PDF for `style_name` from kateri and write it to `output_filename`.""" - data = await kateri.communicator.get_pdf(style=style_name) - print(f">>> writing pdf to {output_filename}", file=sys.stderr) - with output_filename.open("wb") as output: - output.write(data) - # if open: - # subprocess.call(["open", expected["filename"]]) + async def export_pdf(self, style_name: str, output_filename: Path, chart: Optional[ae_backend.chart_v3.Chart] = None): + """Render the map for `style_name` to `output_filename`, via the map-render backend + selected by `AE_REPORT_MAP_RENDERER` (default kateri; `native` = in-process C++ + `export_styled_map`). `chart` is the styled chart; it is required by the native + backend and ignored by the kateri backend (which already holds the chart from the + `style` command).""" + await get_map_renderer().export_pdf(chart=chart, style_name=style_name, output_filename=output_filename) def serum_coverage_output_dir(self, check_existance: bool = False) -> Path | None: """The `serum-coverage/` output directory. With `check_existance`, return it only if diff --git a/py/ae/report/map_renderer.py b/py/ae/report/map_renderer.py new file mode 100644 index 0000000..fb3a4e0 --- /dev/null +++ b/py/ae/report/map_renderer.py @@ -0,0 +1,212 @@ +# P2 milestone J — pluggable map-render backend for the report's per-map PDF step. +""" +ae.report.map_renderer — select how the report renders each antigenic-map PDF. + +The report's map step is exactly "pick an on-chart named style -> get a PDF". This module +abstracts that step behind a small `MapRenderer` seam with two interchangeable backends that +consume the SAME on-chart styling the report already bakes (`c["R"]` named styles + `c["p"]` +base plot-spec): + + - `KateriRenderer` — drive the kateri app over its Unix socket (`set_style` + + `get_pdf`). This is the previous behaviour, now opt-in via the env var. + - `NativeRenderer` — call `ae_backend.map_draw.export_styled_map` in-process: no + subprocess, no socket, Linux-capable, and (P2's payoff) no kateri launch per map. + +The backend is chosen by the env var `AE_REPORT_MAP_RENDERER`: + + AE_REPORT_MAP_RENDERER=native (default / unset) -> NativeRenderer + AE_REPORT_MAP_RENDERER=kateri -> KateriRenderer + +Native is the default report figure renderer (headless, in-process, Linux-capable) after P2 +sign-off. This only affects the report's batch map-PDF step; kateri remains the interactive +viewer (drag/Relax/GUI) and the fallback here via `AE_REPORT_MAP_RENDERER=kateri`. +""" +import os +import sys +import json +import math +import tempfile +from pathlib import Path + +import ae_backend.chart_v3 + +from ae.utils import kateri + +# ====================================================================== + +ENV_VAR = "AE_REPORT_MAP_RENDERER" +DEFAULT_BACKEND = "native" + +# ---------------------------------------------------------------------- + +class MapRenderer: + """Interface for the report's per-map PDF render step: given the styled chart, a named + style, and an output path, write the map PDF. Backends take the SAME inputs so they are + drop-in for each other.""" + + backend_name = "?" + + async def export_pdf(self, chart: ae_backend.chart_v3.Chart, style_name: str, output_filename: Path, width: float = 800.0): + """Render the map for `style_name` (a name in the chart's `c["R"]` block) to + `output_filename` as a PDF, at page `width` points.""" + raise NotImplementedError("override in a MapRenderer backend") + +# ---------------------------------------------------------------------- + +class KateriRenderer(MapRenderer): + """Default backend: request the PDF from a connected kateri over its socket. `chart` is + ignored here because the report has already sent it to kateri (via the `style` command) + before requesting PDFs — this backend only selects the style and collects the bytes, + exactly as the report's map step did before the seam was introduced.""" + + backend_name = "kateri" + + async def export_pdf(self, chart: ae_backend.chart_v3.Chart, style_name: str, output_filename: Path, width: float = 800.0): + """Ask kateri for the PDF of `style_name` and write it to `output_filename`.""" + data = await kateri.communicator.get_pdf(style=style_name, width=width) + print(f">>> [map_renderer.kateri] writing pdf to {output_filename}", file=sys.stderr) + Path(output_filename).write_bytes(data) + +# ---------------------------------------------------------------------- + +class NativeRenderer(MapRenderer): + """Headless in-process backend: render the styled chart with the native C++ renderer + (`ae_backend.map_draw.export_styled_map`), which resolves the same `c["R"]` named style + + `c["p"]` base plot-spec kateri consumes. No kateri process, no socket. `export_styled_map` + reads a chart from an `.ace` file, so the report's in-memory styled chart is written to a + short-lived temp `.ace` for each render.""" + + backend_name = "native" + + async def export_pdf(self, chart: ae_backend.chart_v3.Chart, style_name: str, output_filename: Path, width: float = 800.0): + """Write the styled chart to a temp `.ace` and render `style_name` natively to + `output_filename`.""" + if chart is None: + raise RuntimeError( + f"{self.__class__.__name__} needs the in-memory styled chart, but none was " + f"passed to export_pdf (style {style_name!r}). The native backend renders from " + f"the chart directly rather than from a kateri session.") + fd, tmp_name = tempfile.mkstemp(suffix=".ace", prefix="ae-report-native-") + os.close(fd) + tmp_path = Path(tmp_name) + try: + chart.write(tmp_path) + print(f">>> [map_renderer.native] rendering style {style_name!r} -> {output_filename}", file=sys.stderr) + ae_backend.map_draw.export_styled_map(tmp_path, Path(output_filename), style_name, width, 0) + finally: + try: + tmp_path.unlink() + except FileNotFoundError: + pass + +# ---------------------------------------------------------------------- + +_BACKENDS = { + "kateri": KateriRenderer, + "native": NativeRenderer, +} + +def selected_backend_name() -> str: + """The backend name from `AE_REPORT_MAP_RENDERER` (default `kateri` when unset/empty).""" + return (os.environ.get(ENV_VAR) or DEFAULT_BACKEND).strip().lower() or DEFAULT_BACKEND + +def get_map_renderer() -> MapRenderer: + """Instantiate the `MapRenderer` selected by `AE_REPORT_MAP_RENDERER` (default kateri).""" + name = selected_backend_name() + try: + backend = _BACKENDS[name] + except KeyError: + raise RuntimeError(f"unknown {ENV_VAR}={name!r}; expected one of {sorted(_BACKENDS)}") + return backend() + +def native_selected() -> bool: + """True when `AE_REPORT_MAP_RENDERER=native` — the report's finalising steps (write + `styled.ace`, compute the signature-page mapi viewport) must run kateri-free. When False + (default) the report keeps its exact previous kateri behaviour.""" + return selected_backend_name() == "native" + +# ====================================================================== +# Native, kateri-free replacements for the `export` command's finalising steps. +# +# The per-map PDF step already has a native backend (NativeRenderer, above). But +# CommanderBasic.export finishes with two more kateri round-trips that the two functions +# below replace so an `export` run needs NO kateri process at all: +# +# 1. write_styled_ace() <- kateri export_to_legacy(style) + get_chart() + write +# 2. sig_page_viewport() (mapi) <- kateri set_style(style) + get_viewport() +# ---------------------------------------------------------------------- + +def write_styled_ace(chart: ae_backend.chart_v3.Chart, filename: Path): + """Native replacement for `export`'s trailing `kateri.export_to_legacy(style) + + get_chart() + write(styled.ace)`. + + kateri's `export_to_legacy` runs `plotSpecLegacy().setFrom(currentPlotSpec)` + (map-viewer-data.dart `exportCurrentPlotStyleToLegacy`) — it bakes the *named* semantic + style (`c["R"][style_for_legacy_plot_spec()]`) into the legacy per-point plot spec + (`c["p"]` = per-point palette index + palette). `get_chart` then returns the whole chart + (semantic attributes + `c["R"]` styles + `c["P"]` projections + the freshly-baked + `c["p"]`). + + Everything except that fresh `c["p"]` bake is already present on the in-memory styled + chart, so a plain `chart.write()` reproduces the semantic attributes, all `c["R"]` named + styles and the `c["P"]` projections byte-for-byte. The `c["p"]` legacy plot spec carried + on the chart is the one baked by the *earlier* `prestyle` step — which bakes the SAME + `style_for_legacy_plot_spec()` style (see CommanderBasic.prestyle / .export) — so its + per-antigen clade fills are identical to what a fresh kateri bake would produce (verified + against the kateri-written styled.ace: the only differences are colour-string + normalisation, e.g. `#6D9BC2`->`#6d9bc2` and `#CCCCCC`->`gray80`). The one consumer of + this legacy plot spec, `multiple_circles.legacy_fills`, reads only those per-antigen + fills, so it is served identically. + + Residual (honest gap): this relies on the in-memory chart already carrying a `c["p"]` + baked from `style_for_legacy_plot_spec()` (which the standard download->prestyle->adjust + pipeline guarantees). It does NOT itself re-run the semantic->legacy bake, because that + logic (`setFrom`) lives only in kateri; a native `Chart.semantic_style_to_legacy` binding + (stubbed out in cc/chart/v3/chart.hh) would be needed to bake it in-process.""" + print(f">>> [map_renderer.native] writing styled chart -> {filename}", file=sys.stderr) + chart.write(Path(filename)) + +def sig_page_viewport(chart: ae_backend.chart_v3.Chart, used_viewport) -> list[float]: + """Native replacement for the `kateri.set_style(style) + get_viewport()` round-trip in + `ChartModifier.export_mapi_for_signature_pages`. + + kateri returns three things and the report combines them into the sig-page viewport: + * `used` = the style's own viewport `[x, y, size]` (the report set this via + `self.viewport()`), passed in here as `used_viewport`; + * `native` = the projection viewport AFTER kateri's roundAndRecenter, i.e. a box + whose width/height are `ceil(hull_span + 1)` of the TRANSFORMED + layout (viewport.dart roundAndRecenter); + * `native_center` = the centre of the transformed-layout hull BEFORE rounding + (viewport.dart `layoutCenter2`). + The report then computes (commander/chart_modifier): + x = -native_w/2 + used_x + native_center_x + y = -native_h/2 + used_y + native_center_y + size = used_size + which this function reproduces from the transformed layout (raw layout `c["P"][0]["l"]` + put through the projection transformation `c["P"][0]["t"]`, convention + `tx = x*a + y*c`, `ty = x*b + y*d` per cc/chart/v3/transformation.hh). Verified to + reproduce kateri's `sp.mapi` viewport to full float precision across H1/H3/B labs.""" + ux, uy, us = float(used_viewport[0]), float(used_viewport[1]), float(used_viewport[2]) + data = json.loads(chart.export())["c"] + projections = data.get("P", []) + if not projections: + return [ux, uy, us] + proj = projections[0] + layout = proj.get("l", []) + a, b, c, d = proj.get("t", [1.0, 0.0, 0.0, 1.0])[:4] + xs: list[float] = [] + ys: list[float] = [] + for co in layout: + if co and len(co) >= 2 and co[0] is not None and co[1] is not None and co[0] == co[0] and co[1] == co[1]: + x, y = co[0], co[1] + xs.append(x * a + y * c) + ys.append(x * b + y * d) + if not xs: + return [ux, uy, us] + cx = (min(xs) + max(xs)) / 2.0 + cy = (min(ys) + max(ys)) / 2.0 + rw = math.ceil((max(xs) - min(xs)) + 1.0) + rh = math.ceil((max(ys) - min(ys)) + 1.0) + return [cx - rw / 2.0 + ux, cy - rh / 2.0 + uy, us] + +# ====================================================================== diff --git a/tools/p2-fidelity/README.md b/tools/p2-fidelity/README.md new file mode 100644 index 0000000..cf76af5 --- /dev/null +++ b/tools/p2-fidelity/README.md @@ -0,0 +1,73 @@ +# P2 fidelity harness — the renderer scoreboard + +`fidelity.py` is the reusable pixel-diff harness that every P2 map-renderer milestone uses to +prove itself. Given a manifest of `(chart, style, [golden], [ad-ref])` entries it renders each +map with the **native** renderer (`ae_backend.map_draw.export_styled_map`) and pixel-compares +it against (a) the **kateri** golden PDF and (b) an **AD** single-map reference where one is +isolable — emitting a per-map AE (differing-pixel) table at both `-fuzz 30%` and strict. + +## What it does per map + +1. **render** native → PDF: `export_styled_map(chart, out.pdf, style, width, projection)` +2. **rasterise** native + each reference PDF to equal-size PNGs: `pdftoppm -png -scale-to SIZE` +3. **compare** native vs each reference: `magick compare -metric AE [-fuzz 30%]` → AE pixel count +4. **report**: per-map table to stdout + `scoreboard.csv` + `scoreboard.md` in the out-dir, + plus optional `native|kateri[|AD]` montages (`--montage`). + +`AE` is the count of differing pixels; `%` is `AE / (SIZE·SIZE)`. `-fuzz 30%` tolerates small +colour/AA deltas so it isolates *structural/content* differences from anti-aliasing noise; +strict is the raw count. + +## Invocation + +```bash +# ae_backend lives in the frozen render-spike build; point --build-dir at it (no rebuild). +python3.14 tools/p2-fidelity/fidelity.py MANIFEST.json \ + --build-dir /path/to/ae-p2-render-spike/build-py314 \ + --out-dir /tmp/p2-fidelity-run \ + --size 800 --montage +``` + +Flags: `--build-dir DIR` (dir with `ae_backend*.so`; else `$AE_BUILD`, else an already-importable +`ae_backend` on `PYTHONPATH`, else `/build-py314`), `--out-dir DIR` (default a scratch dir), +`--size PX` (raster scale-to, default 800), `--width` / `--projection` (native render overrides), +`--montage` (write side-by-side PNGs), `--only SUBSTR` (filter by label), `--timeout SEC` +(per-command bound, default 120). + +Requires: `python3.14` with `ae_backend`, `pdftoppm` (poppler), ImageMagick 7 (`magick`), +and `timeout` — every external command is wrapped in `timeout` so a hung tool cannot stall a run. + +## Manifest format + +JSON. Either a top-level object with config + a `maps` list, or a bare list of entries. See +[`manifest.example.json`](manifest.example.json) for the full annotated schema. Essentials: + +| key | scope | meaning | +|-----|-------|---------| +| `chart` | entry (req) | path to a styled `.ace` (outside the repo) | +| `style` | entry (req) | a `c["R"]` front-style name: `clades-v1`, `serology`, `ts-2025-08`, `info-clades-v1`, … | +| `golden` | entry | kateri reference PDF path; `"auto"` derives `/out.1.