From fdd554f56bb7ca9c0d435424551b92ab44f64e17 Mon Sep 17 00:00:00 2001 From: Patrick Leiverkus Date: Sun, 26 Jul 2026 16:58:33 +0200 Subject: [PATCH] docs: name the backend axis correctly and precise the v0.9 blocker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things went stale and were found by running the pipeline on real data for the first time. "SAM vs. Cellpose" is not the axis. CellposeSegmenter runs Cellpose-SAM v4 (Pachitariu et al. 2025), which is itself SAM-backboned — the two learned backends share an architecture and differ in what they were trained on. The real axis is zero-shot generic vs. domain-finetuned, and the companion plan's H_A already states it that way: a specialist fine-tuned on <= 300 hand-labelled stones against zero-shot SAM. The paper's literature guide says the same thing from the other side, calling ImageGrains "Cellpose for stones". v0.9's blocker is annotation, not capture. A real orthophoto and DEM exist (paper/input/data/Export/, Tiberias 2023-03-10) and the pipeline has now been run on them end-to-end. What is missing is ground truth — no reference stone outlines anywhere in the paper repo — so no metric can be scored. H_A bounds the work at <= 300 instances, which makes it a schedulable task rather than a wait for a season. Add the freeze-first task that follows from this: looking at model output on a scene before its hold-out status is designated spends the scene. v1.0's blocker is restated as the v0.9 verdict, since the capture it named now exists. Co-Authored-By: Claude Opus 5 --- ROADMAP.md | 25 +++++++++++++++++++++---- docs/architecture.md | 9 +++++++-- 2 files changed, 28 insertions(+), 6 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index a099ed5..d0f72c7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -39,8 +39,14 @@ no GPU, no model download, no database required. - [x] Backend selection via configuration (`STRUCTURA_2D_BACKEND`, `make_segmenter`). - [x] Geometry metric utilities (over-/under-segmentation rate, a/b-axis error; plus IoU/matching, precision/recall/F1) in `structura.metrics`. -- Data-dependent: model fine-tuning and the SAM-vs-Cellpose verdict belong to - the evaluation milestone (v0.9), not here. +- Data-dependent: model fine-tuning and the backend verdict belong to the + evaluation milestone (v0.9), not here. +- The axis is **zero-shot generic vs. domain-finetuned**, not "SAM vs. Cellpose". + `CellposeSegmenter` runs Cellpose-SAM v4 (Pachitariu et al. 2025), which is + itself SAM-backboned — the two learned backends share an architecture and + differ in what they were trained on. The companion plan's H_A states it that + way: a specialist fine-tuned on ≤ 300 hand-labelled stones against zero-shot + SAM. ### v0.4.0 — 2.5D track ✅ *(released 2026-06-16)* *(Sub-study B)* - [x] `dem.relief` derivatives (hillshade, slope, curvature, local relief model, @@ -115,12 +121,23 @@ intended delivery path. - [ ] Validate the archaeological usefulness of the semantic ortho — Structura's evaluation, not Effigies'. -### v0.9.0 — Evaluation harness ⛔ *needs real trench data* +### v0.9.0 — Evaluation harness ⛔ *needs **annotated** trench data* + +The blocker is annotation, not capture. A real orthophoto and DEM exist +(`paper/input/data/Export/`, Tiberias 2023-03-10) and the pipeline has been run +on them end-to-end. What does not exist is ground truth: no reference stone +outlines anywhere in the paper repo, so no metric can be scored. H_A puts a +bound on it — ≤ 300 hand-labelled instances — which makes this a schedulable task +rather than a wait for a season. + +- [ ] Ground-truth annotation on frozen hold-out scenes (**freeze the scene IDs + first** — plan Task 1.1; looking at model output on a scene before it is + designated spends it). - [ ] Reproducible evaluation scripts producing the paper's metrics (AP@IoU; completeness/correctness/quality + clDice/APLS; LoD95%). - [ ] Wire results into `paper/output/data-analysis/`. -### v1.0.0 — Production release ⛔ *needs real trench data* +### v1.0.0 — Production release ⛔ *needs the v0.9 verdict* - [ ] One full trench processed end-to-end. - [ ] DB default locked; stable public API. - [ ] Documentation complete; first Zenodo DOI minted. diff --git a/docs/architecture.md b/docs/architecture.md index 2eb0650..d64eecb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -143,7 +143,12 @@ The active sink is chosen by `STRUCTURA_SINK` (`file` | `postgis` | `api`). PostGIS directly. Awaiting confirmation. The record explains why the original PostGIS-vs-API framing was incomplete: the API does not exist, and neither sink can land geometry that has no context yet. -- **2D model choice:** SAM vs. Cellpose vs. classical CV — to be decided by the - comparative evaluation (blocked on real excavation data). +- **2D model choice:** **zero-shot generic vs. domain-finetuned**, with the + classical watershed as the deterministic floor — to be decided by the + comparative evaluation. Not "SAM vs. Cellpose": `CellposeSegmenter` runs + Cellpose-SAM v4, which is SAM-backboned, so the two learned backends share an + architecture and differ in training data. Blocked on **annotated** excavation + data, not on capture — a real orthophoto and DEM exist and have been processed; + ground-truth stone outlines do not. *(The intake-layout question that stood here is resolved — see [Intake](#intake-structuraintake) above.)*