Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,27 @@ STRUCTURA_GPU=false
# so on a real orthophoto it filters nothing. 0.0025 (25 cm²) is a usable start
# for stones; raise it until the speckle disappears.
STRUCTURA_MIN_AREA=1e-4
# Drop 2D polygons ABOVE this area. Unset = no cap, which lets every backend's
# background mask through: on a real 815 m² trench SAM emitted one polygon of
# 634 m² — the full raster rectangle with 1790 holes punched out — and the
# watershed one of 29 m². Set it to the largest thing you are looking for.
STRUCTURA_MAX_AREA=

# --- Cellpose backend only ---
# Expected object size in pixels. Unset lets Cellpose estimate it, using an
# estimate tuned for cells; on stones it can settle on the large blocks and drop
# everything smaller. Set it explicitly if recall looks low.
STRUCTURA_CELLPOSE_DIAMETER=
# The recall knob: lower admits more (and fainter) instances. 0.0 is the Cellpose
# default, not a tuned value.
STRUCTURA_CELLPOSE_CELLPROB=0.0
STRUCTURA_CELLPOSE_FLOW=0.4
# Path to a fine-tuned Cellpose checkpoint. Unset = Cellpose's generalist default
# (cpsam_v2). The domain-relevant option is ImageGrains 2.0 (Mair et al. 2026),
# Cellpose-SAM fine-tuned on sediment grains: zenodo.org/records/15728186, file
# IG2_full_set_cp_SAM (1.2 GB, CC-BY-4.0). Its 26 MB checkpoints are Cellpose-2
# U-Nets and will NOT load here.
STRUCTURA_CELLPOSE_MODEL=

# --- 2.5D track (DEM wall tracing): gap-bridging tolerance in world units (m) ---
STRUCTURA_GAP_BRIDGE_M=0.3
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- **`STRUCTURA_MAX_AREA` — an upper bound on 2D polygon area.** Every backend
emits a *background* mask, and without a cap it reaches the output and swamps
everything in QGIS. On a real 815 m² trench, SAM produced a single polygon of
**634 m²** — the full raster rectangle, matching the extent to 0.000 m on all
four edges, with 1790 holes punched out where it had segmented something else;
the Otsu watershed produced one of 29 m². With the cap at 5 m² the watershed run
goes from 109 features to 108, and the largest survivor drops from 419.8 m² to
0.139 m².
- **Cellpose tuning is reachable from configuration**: `STRUCTURA_CELLPOSE_DIAMETER`,
`STRUCTURA_CELLPOSE_CELLPROB` and `STRUCTURA_CELLPOSE_FLOW`. `cellprob_threshold`
did not exist on the segmenter at all, and it is the recall knob — the one that
matters when the backend returns few but correct instances. `diameter` existed
but, like `min_area` before it, was never passed by `make_segmenter`.
- **`STRUCTURA_MIN_AREA` — the 2D polygon area filter is now configurable.** All
three segmenters already accepted `min_area`, but `make_segmenter` never passed
it, so the only way to change it was to edit the source. On a real orthophoto
Expand Down
95 changes: 95 additions & 0 deletions docs/adr/0002-multiscale-2d.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# ADR-0002 — The 2D track is scale-dependent; one diameter is not a superset

## Status

**Proposed** — 2026-07-26. Affects the v0.3 backend defaults and the shape of the
v0.9 evaluation.

## Context

Cellpose's `diameter` was assumed to be a sensitivity knob: raise it and the
backend finds more, keeping what it already had. Measured on the Tiberias M10
extent (815 m², 0.526 cm/px, `min_area` 25 cm², `max_area` 5 m², `cellprob` 0.0),
that is false.

Object counts rise with diameter — 328 at Cellpose's own estimate (which lands at
about 30 px), 405 at 60 px, 533 at 120 px. But matching the instances against each
other with `structura.metrics.match_instances` shows they are largely **different
objects**:

| | shared | only in A | only in B | of A recovered |
|---|---|---|---|---|
| auto (30 px) vs 60 px | 151 | 177 | 254 | 46 % |
| 60 px vs 120 px | 217 | 188 | 316 | 54 % |
| **auto vs 120 px** | **130** | **198** | **403** | **40 %** |

This is not boundary jitter. Relaxing the IoU threshold from 0.5 to 0.3 moves the
auto-vs-120 px overlap from 130 to 136 — the instances genuinely differ rather
than being the same stones drawn slightly differently.

What is lost when the diameter rises is systematically **smaller**:

| | n | median | p10 |
|---|---|---|---|
| in both | 130 | 1042 cm² | 272 cm² |
| only at auto / 30 px | 198 | **511 cm²** | **40 cm²** |
| only at 120 px | 403 | 784 cm² | 196 cm² |

So `diameter` selects a **scale band**. Raising it trades small detections for
different mid-sized ones rather than adding to the set.

The union across all four sampled diameters, deduplicated at IoU 0.5, holds **856
objects** against 533 for the best single run — **61 % more than any one
setting can produce**.

The `pretrained_model` axis behaves the same way, only more strongly:
ImageGrains 2.0, fine-tuned on sediment grains, returns 3130 objects at a median
of 138 cm² where the generalist `cpsam_v2` returns 393 at 866 cm². Both are
Cellpose-SAM; the fine-tuning material moves the scale band, it does not simply
raise recall.

## Decision

Treat the 2D track's scale as a **sweep dimension, not a setting**.

1. Do **not** pick a default `diameter` from these numbers. There is no value
that dominates, and "most objects" is not a quality criterion without ground
truth.
2. Add a multi-scale mode to the 2D track: run a configured set of diameters and
merge the results, mirroring `dem.relief.multiscale_relief`, which already
does exactly this for the 2.5D track.
3. The merge needs more than IoU deduplication. Containment and partial overlap
have to be resolved — one stone found whole at 120 px and as two halves at
30 px survives both passes under an IoU-only rule, which is why the 856 above
is an upper bound on distinct detections rather than a count of stones.
4. The v0.9 evaluation must state which scale configuration it scores. A single
`(model, diameter)` pair is one sample from a family, and reporting it as
"the backend's output" would overstate what was measured.

## Consequences

**Accepted costs.**

- A multi-scale pass costs roughly the sum of its scales. At 15–30 s per scale on
an RTX A4000 that is still under two minutes for a trench, so the cost is real
but small.
- The merge rule becomes a piece of project-specific logic that has to be
justified and tested. It is not a library call.
- Until it exists, any single-setting output — including everything produced on
2026-07-26 — under-reports what the backend can find.

**What this buys.**

- It stops the search for a "correct" diameter, which the measurements show does
not exist.
- It makes the H_A comparison honest: the fine-tuned specialist and zero-shot SAM
are both scale-dependent, and comparing one arbitrary setting of each would
measure the settings as much as the models.
- The 61 % headroom is available without any annotation.

**Open.**

- Whether the extra detections at small diameters are stones or ground texture.
Only ground truth answers this, and it may turn out that the narrow band is the
right one — in which case the decision becomes "pick a band deliberately"
rather than "merge them", but it would still be a decision made on evidence.
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ reaches the excavation database does.
| ADR | Title | Status |
|---|---|---|
| [0001](0001-vector-sink.md) | How Structura reaches the excavation database | Proposed |
| [0002](0002-multiscale-2d.md) | The 2D track is scale-dependent; one diameter is not a superset | Proposed |
9 changes: 8 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,14 @@ implement the `Segmenter` protocol (`segment(ortho_path) -> list[Feature]`):
(`segment-geospatial`): runs on the (tiled) georeferenced ortho and writes a
label raster, which is read back through the shared path.
- `CellposeSegmenter` — **Cellpose-SAM (v4)** instance masks
(`CellposeModel.eval`).
(`CellposeModel.eval`). `STRUCTURA_CELLPOSE_MODEL` points it at a fine-tuned
checkpoint; ImageGrains 2.0 (sediment grains) is the domain-near option.

**The output is scale-dependent.** `diameter` selects a size band rather than a
sensitivity: on a real trench, the instances found at 30 px and at 120 px overlap
by only 40 %, and the union across four diameters is 61 % larger than the best
single run. See [ADR-0002](adr/0002-multiscale-2d.md) — this is why no default
diameter is set, and why the v0.9 evaluation has to state its scale configuration.

All three end the same way: an integer instance-label mask →
`segmentation._common.label_mask_to_features` → `geo.mask_to_polygons` →
Expand Down
28 changes: 28 additions & 0 deletions src/structura/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ def _load_dotenv(path: Path) -> None:
os.environ.setdefault(key.strip(), value.strip())


def _opt_float(name: str) -> float | None:
"""Read an optional numeric setting. Unset *and* empty both mean "no value",
so a commented-out or blanked line in `.env` behaves the way it looks."""
raw = os.environ.get(name, "").strip()
return float(raw) if raw else None


@dataclass(slots=True)
class Settings:
input_dir: Path
Expand All @@ -32,6 +39,18 @@ class Settings:
# The useful value depends on ground resolution and on what counts as a find:
# at 0.5 cm/px the default 1e-4 m² is ~4 px, i.e. effectively no filter.
min_area: float
# 2D track: drop polygons *above* this area. Unset means no cap, which lets
# every backend's background mask through — SAM's covered 78 % of a real
# trench. Set it to the largest thing you are looking for.
max_area: float | None
# Cellpose backend only. `diameter=None` lets Cellpose estimate object size;
# `cellprob_threshold` is the recall knob (lower admits more instances).
cellpose_diameter: float | None
cellpose_flow_threshold: float
cellpose_cellprob_threshold: float
# Path to a fine-tuned Cellpose checkpoint; unset uses Cellpose's own
# generalist default. See ImageGrains 2.0 for a sediment-grain model.
cellpose_model: str | None
# 2.5D track: gap-bridging tolerance (world units) for wall tracing
gap_bridge_m: float
# File sink (default)
Expand Down Expand Up @@ -59,6 +78,15 @@ def from_env(cls, dotenv: str | Path = ".env") -> "Settings":
segmentation_backend=os.environ.get("STRUCTURA_2D_BACKEND", "classical"),
gpu=os.environ.get("STRUCTURA_GPU", "").lower() in ("1", "true", "yes"),
min_area=float(os.environ.get("STRUCTURA_MIN_AREA", "1e-4")),
max_area=_opt_float("STRUCTURA_MAX_AREA"),
cellpose_diameter=_opt_float("STRUCTURA_CELLPOSE_DIAMETER"),
cellpose_flow_threshold=float(
os.environ.get("STRUCTURA_CELLPOSE_FLOW", "0.4")
),
cellpose_cellprob_threshold=float(
os.environ.get("STRUCTURA_CELLPOSE_CELLPROB", "0.0")
),
cellpose_model=os.environ.get("STRUCTURA_CELLPOSE_MODEL") or None,
gap_bridge_m=float(os.environ.get("STRUCTURA_GAP_BRIDGE_M", "0.3")),
output_path=Path(
os.environ.get("STRUCTURA_OUTPUT_PATH", "./data/output/features.gpkg")
Expand Down
27 changes: 21 additions & 6 deletions src/structura/geo.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,27 @@ def read_raster(path: Path) -> tuple[Any, Any, str]:


def mask_to_polygons(
mask: Any, transform: Any, crs: str, *, min_area: float = 0.0
mask: Any,
transform: Any,
crs: str,
*,
min_area: float = 0.0,
max_area: float | None = None,
) -> list[Any]:
"""Vectorise a boolean/label mask into georeferenced Shapely polygons.

Used by the 2D track to turn SAM/Cellpose/classical masks into stone/surface
outlines. ``transform`` carries the pixel size, so the returned geometries —
and therefore ``min_area`` (a threshold in world units squared) — are in the
raster CRS. ``crs`` is accepted for interface symmetry; the CRS is attached
later at the :class:`~structura.models.Feature` level.
and therefore ``min_area`` / ``max_area`` (thresholds in world units squared)
— are in the raster CRS. ``crs`` is accepted for interface symmetry; the CRS
is attached later at the :class:`~structura.models.Feature` level.

``max_area`` discards masks that are too large to be a find. Every backend
produces them: SAM's automatic generator emits a *background* mask covering
everything it did not segment — on a real trench that was the full raster
rectangle, 78 % of the extent, with 1790 holes punched out — and the Otsu
watershed does the same when its global threshold splits foreground from
ground. Without an upper bound these survive and swamp the output in QGIS.

Background pixels (label ``0``) are skipped. Each polygon is repaired with a
zero-width buffer to fix any self-touching rings from the marching-squares
Expand All @@ -52,8 +64,11 @@ def mask_to_polygons(
if value == 0:
continue
geom = shape(geom_dict).buffer(0)
if geom.area >= min_area:
polygons.append(geom)
if geom.area < min_area:
continue
if max_area is not None and geom.area > max_area:
continue
polygons.append(geom)
return polygons


Expand Down
16 changes: 13 additions & 3 deletions src/structura/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,21 @@
def make_segmenter(settings: Settings) -> Segmenter:
backend = settings.segmentation_backend
if backend == "classical":
return ClassicalSegmenter(min_area=settings.min_area)
return ClassicalSegmenter(
min_area=settings.min_area, max_area=settings.max_area
)
if backend == "sam":
return SamSegmenter(min_area=settings.min_area)
return SamSegmenter(min_area=settings.min_area, max_area=settings.max_area)
if backend == "cellpose":
return CellposeSegmenter(gpu=settings.gpu, min_area=settings.min_area)
return CellposeSegmenter(
gpu=settings.gpu,
diameter=settings.cellpose_diameter,
flow_threshold=settings.cellpose_flow_threshold,
cellprob_threshold=settings.cellpose_cellprob_threshold,
pretrained_model=settings.cellpose_model,
min_area=settings.min_area,
max_area=settings.max_area,
)
raise ValueError(f"Unknown 2D backend: {backend!r}")


Expand Down
11 changes: 8 additions & 3 deletions src/structura/segmentation/_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,19 @@ def label_mask_to_features(
segmenter: str,
source_raster: Path | str,
min_area: float = 1e-4,
max_area: float | None = None,
feature_type: FeatureType = FeatureType.STONE,
) -> list[Feature]:
"""Vectorise an instance-label ``mask`` into 2D-track ``Feature`` objects.

``mask`` is an integer array (label 0 = background). ``min_area`` is in world
units squared (CRS); ``transform``/``crs`` come from the source raster.
``mask`` is an integer array (label 0 = background). ``min_area`` and
``max_area`` are in world units squared (CRS); ``transform``/``crs`` come
from the source raster. See :func:`structura.geo.mask_to_polygons` for why an
upper bound matters.
"""
geoms = geo.mask_to_polygons(mask, transform, crs, min_area=min_area)
geoms = geo.mask_to_polygons(
mask, transform, crs, min_area=min_area, max_area=max_area
)
return [
Feature(
feature_type=feature_type,
Expand Down
37 changes: 35 additions & 2 deletions src/structura/segmentation/cellpose.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,41 @@ def __init__(
gpu: bool = False,
diameter: float | None = None,
flow_threshold: float = 0.4,
cellprob_threshold: float = 0.0,
pretrained_model: str | Path | None = None,
min_area: float = 1e-4,
max_area: float | None = None,
) -> None:
self.gpu = gpu
# Expected object size in pixels; None lets Cellpose estimate it. The
# estimate is tuned for cells — on stones it can settle on the large
# blocks and drop everything smaller, so it is worth setting explicitly.
self.diameter = diameter
self.flow_threshold = flow_threshold
# The recall knob: lower admits more (and fainter) instances. 0.0 is the
# Cellpose default, not a tuned value.
self.cellprob_threshold = cellprob_threshold
# Path to a fine-tuned checkpoint. Unset uses Cellpose's own default
# (`cpsam_v2`), a generalist cell model. The domain-relevant alternative
# is ImageGrains 2.0 (Mair et al. 2026), Cellpose-SAM fine-tuned on
# sediment grains in orthophotos — same architecture, so it loads here
# unchanged. Its older 26 MB checkpoints are Cellpose-2 U-Nets and will
# *not* load into Cellpose 4.
self.pretrained_model = pretrained_model
self.min_area = min_area
self.max_area = max_area

def cellpose_kwargs(self) -> dict:
"""Constructor arguments for ``CellposeModel``.

``pretrained_model`` is omitted when unset so Cellpose picks its own
default, rather than being passed ``None`` — the mistake that kept the
SAM backend from ever running.
"""
kwargs: dict = {"gpu": self.gpu}
if self.pretrained_model is not None:
kwargs["pretrained_model"] = str(self.pretrained_model)
return kwargs

def segment(self, ortho_path: Path) -> list[Feature]:
import numpy as np # noqa: PLC0415
Expand All @@ -42,9 +71,12 @@ def segment(self, ortho_path: Path) -> list[Feature]:
# rasterio (bands, rows, cols) -> (rows, cols, channels), keep up to RGB.
img = np.transpose(np.asarray(array), (1, 2, 0))[:, :, :3]

model = models.CellposeModel(gpu=self.gpu)
model = models.CellposeModel(**self.cellpose_kwargs())
masks, _, _ = model.eval(
img, diameter=self.diameter, flow_threshold=self.flow_threshold
img,
diameter=self.diameter,
flow_threshold=self.flow_threshold,
cellprob_threshold=self.cellprob_threshold,
)

# masks is an instance-label array (0 = background).
Expand All @@ -55,4 +87,5 @@ def segment(self, ortho_path: Path) -> list[Feature]:
segmenter=self.name,
source_raster=ortho_path,
min_area=self.min_area,
max_area=self.max_area,
)
4 changes: 3 additions & 1 deletion src/structura/segmentation/classical.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ class ClassicalSegmenter:

name = "classical"

def __init__(self, min_area: float = 1e-4) -> None:
def __init__(self, min_area: float = 1e-4, max_area: float | None = None) -> None:
# Minimum polygon area to keep, in world units squared (CRS).
self.min_area = min_area
self.max_area = max_area

def segment(self, ortho_path: Path) -> list[Feature]:
import numpy as np # noqa: PLC0415
Expand Down Expand Up @@ -69,4 +70,5 @@ def segment(self, ortho_path: Path) -> list[Feature]:
segmenter=self.name,
source_raster=ortho_path,
min_area=self.min_area,
max_area=self.max_area,
)
Loading