gen1 may not resolve ground under dense grass (reads as false erosion)

[awickert]

The problem. gen1's per-cell ground elevation is the MEDIAN of its CSF-classified
ground returns. Under a dense sward, returns that stop in the grass enter that class and
the median floats above true ground. gen2 has roughly 12x the ground-return density
(5.78 vs 0.49 pts/m^2), so far more of its shots reach the real surface through the same
grass. The DoD is gen2 - gen1, so the result reads as false erosion.

This is a SAMPLING argument, not a leaf-state one. It is independent of, and opposite in
sign to, the leaf-on canopy bias that lifts gen2 and reads as false deposition
(analysis/ngv.py, analysis/ngv_correct_dod.py). Two epochs, two mechanisms, opposite
signs -- which is the likeliest reason a single vegetation correction overshot at Elba.

What is measured (analysis/gen1_grass_lift.py, commit 6e41db1; gen1-INTERNAL,
p50 - p10 of each cell's own gen1 ground returns from gen1_csf_angles.npz):

  group                        ncell   p50-p10   slope   cover    DoD mm
  floodplain, cover<0.15       38894      96.0     5.4   0.010     +18.8
  upland,     cover<0.15       99555      60.2     3.8   0.000      -5.2

  floodplain EROSION cells  n=12,789  lift 220.0 mm  slope 10.0  cover 0.170  DoD -188.6
  floodplain, not eroding  n=114,724  lift 106.7 mm  slope 13.8  cover 0.230  DoD  +28.5

Open floodplain against open upland: 96.0 vs 60.2 mm, at near-zero canopy in both and
similar low slope -- meadow sward against cropland. The cells detected as erosion carry
twice the lift while being FLATTER and LESS canopied, so neither intra-cell relief nor
leaf-on explains them; both would push the other way. The lift, 220 mm, is about the
size of the apparent erosion, -189 mm.

Independent support: Google Earth shows the eroding floodplain as open meadow in
September 2008 and a dense sapling thicket in April 2021.

Per-cell lift grid: data/derived/elba_fulldensity/gen1_lift_p50_p10.npy.

Two ways forward, and the choice is open.

1. Correct gen1. Replace the per-cell median with a lower statistic. p10 is not
   proposed -- it was used to measure the spread and is an unjustified estimator.
   A defensible replacement needs either calibration against surveyed 2008 control on
   grass, or a floor derived from return count and footprint. Whatever is chosen must be
   applied the same way to both epochs, or its effect on gen2 measured, or it manufactures
   a difference.

2. Blank the affected cells. Mask cells whose gen1 lift exceeds a threshold and report
   them as no-data rather than as change. Cheaper, honest, and loses coverage exactly
   where the grass is thickest. The threshold is still a choice that has to be justified.

Caution before any tile-wide version. The lift also grows with intra-cell relief. The
flat floodplain is a clean control precisely because relief is low there; a global
correction needs the slope dependence separated first.

Still to do. Confirm the high-lift cells coincide with the 2008 open meadow; choose
between correction and masking; re-run the floodplain DoD against whichever is chosen.

---

Recorded in the repo at TODO.md.

---

Correction, 2026-08-31 — p50 - p10 is a SPREAD, not a BIAS

Raised by the ground_control session's check
(ground_control/REPLY_TO_CROSS_CHECK.md, run_grass_lift_check.py), and it is right.

The diagnostic above measures how far a cell's gen1 ground returns are spread upward from
their own low decile. It does not measure median - true ground, which is the actual
bias. The two are related but not equal, and nothing here pins the constant of
proportionality. Treat the numbers as evidence that something lifts gen1's returns in open
floodplain more than in open upland, not as the size of the correction.

Measured at the 8 open marks the gen1 datum rests on, detrended on the local order-2
surface, vendor class 2:

  median p50-p10 at the marks       40.9 mm   (range 37.9 - 55.1, n = 8)
  reference: open upland, per cell  60.2 mm
  marks minus reference            -19.3 mm

Their statistic is per disc, mine per cell, so this is indicative rather than a like-for-like
difference.

What this does NOT overturn. Their argument that the lift is "part of what is being
calibrated" applies to the datum constant c1, which is measured at marks against the same
median-of-ground-returns surface the DoD runs on, and so absorbs the lift at those marks.
It does not apply to the DoD. DoD = z2 - z1, the two epochs sample the same sward with
~12x different ground-return density, and a constant cannot remove a spatially varying
difference in how far each epoch's median floats. The DoD exposure stands.

What is still needed: a measurement of median - true ground on grass, against
something that is not lidar.

🤖 Generated with Claude Code