[Paullo9]

Requested by Paul · project thread

Part of #8
Part of #15

Stacked on issue-15-pileup-explained (the sum and escape peaks as explained counts for the proposer). This PR's base is that branch; retarget to main once it merges. It is stacked because PooledQuantification.swift is already in two open PRs.

What changes

The pooled quantification fitted one Gaussian per line and no sum (pile-up) peaks, exSpy's model. FitPolicy.sumPeaks (SumPeakRule, default .whenSignificant) now tests the strongest line's sum peak with itself alone (Al K-alpha + Al K-alpha in an Al alloy). If its pile-up coefficient is 3 sigma above 0, the fit keeps the sum peaks of the four strongest lines (EDSSumEscapeFit). Sigma is scaled by sqrt(max(chi2_red, 1)) of that fit. If the test fails, the result is the old fit, bit for bit.

• Why the self pair is the test. On a synthetic spectrum with no pile-up and Cu, Ga and Ge left off the list, the four-parent model kept c at 48 sigma, because Ni K-alpha + O K-alpha sits on Cu K-alpha. The Si self pair read 0.
• The footer names the decision and its number. When the sum peaks are kept, the footer and the warnings also name the alpha lines within one FWHM of the self pair: Ar K-alpha and Ag L-alpha under Al+Al. A real line there is read as pile-up.
• The exSpy-equivalent path stays as a switch: .off (FitPolicy.exspyEquivalent). A caller that passes sumEscape keeps full control. The golden scenarios pin the explicit off model; their numbers are byte-identical, and only the two new nil fields are added.
• Refits. The unlisted-line and fit-range refits repeat the reported decision. The proposer's axis refinement runs without sum peaks.
• The Al K-alpha tail is not part of the default; it is a detector calibration. Fitted together with the sum peaks on the demo's Poisson pools (256 to 2 897 px), it read 0 to a third of the planted fraction and left Mg K-alpha up to 25 % low. Measured once with TailEstimate on a high-count spectrum and typed into FitPolicy.lineTail, it takes Mg K-alpha to within 5 %.

Numbers against planted truth on synthetic data

The truth is the simulation recipe of sim_edx.py v2.0 (mac4DSTEM demo-edx, public), not a measurement, so every result stays validation: "none". The pools come from tools/edx-sim/demo_pools.py and are stored in the new fixture edx-demo-pools.json; the kit's eds-absorption-truth.json is the sixth pool. Setup: true k typed, truth geometry, absorption on, axis locked.

Si planted counts include the Si internal-fluorescence peak, the kit's convention. The "before" columns are the .off fit, which equals the old default bit for bit.

Si K-alpha, net against planted, noise-free

pool	before	after
L5 (recipe A, highest rate)	-57.2 %	-7.9 %
L6 (precipitate)	-8.4 %	-1.2 %
L7 (pure Mg5Si6)	-0.2 %	-0.0 %
M_A (recipe A map, 1217 px)	-14.9 %	-3.6 %
M_matrix (2897 px)	-16.6 %	-3.9 %
absorption-truth fixture	-18.2 %	-4.0 %

Cu K-alpha, read against planted, noise-free

pool	before	after
L5	0.42	0.97
L6	0.00	0.92
L7	0.40	0.93
M_A	1.00	0.99
M_matrix	0.98	0.99

On the L6 Poisson counts Cu reads 0.90 of planted, against 0 before.

Mg K-alpha, net against planted, noise-free

pool	before	sum peaks	sum peaks + calibrated tail (f 0.0095, 0.073 keV)
L5	-24 %	-22 %	+3.2 %
L6	+0.5 %	+0.1 %	+1.1 %
L7	+0.1 %	+0.1 %	+0.8 %
M_A	-32 %	-32 %	+2.6 %
M_matrix	-45.5 %	-45 %	-14.7 % (not explained; seven pooled recipes and thicknesses)

Reduced chi2 on the Poisson counts: L5 6.36 to 0.70, L6 31.3 to 1.08, L7 3.30 to 0.99, M_A 2.11 to 0.83, M_matrix 4.95 to 0.98. The self pair passes on every pool's Poisson counts at z = 5.1 (L6) to 35.5 (M_matrix).

At% on the Poisson counts (Si target includes the internal fluorescence):

• L6: Mg 23.55 (truth 23.03), Si 27.98.
• L7: Mg 45.27 / Si 54.73 (truth 45.45 / 54.55).

Absorption-truth pin: the Si bias moves from -0.203 +- 0.040 to -0.067 +- 0.051. The old pin stays, on the .off fit.

Unlisted-line check (#9): with the default, Ar and Ag are not named on L5, M_A or M_matrix. The Al+Al sum is a modelled peak and is explained to the proposer.

Bounds are measured regression bounds on fixed inputs, not derived limits: Si 5 % (10 % on L5), Cu 0.85 at high rate, chi2_red 1.2.

Migration

PooledQuantifier.run now fits sum peaks by default when the data support them, so nets, at% and wt% change on spectra with pile-up. To keep the old numbers, pass either:

• a policy with fit.sumPeaks = .off (or FitPolicy.exspyEquivalent), or
• sumEscape: SumEscapeModel().

A stored QuantificationPolicy without the new field decodes as .off, and its footer names the non-default policy.

Test evidence

• swift test -c release --parallel --num-workers 1 -j 1 --skip SpectroKitParityTests: green, 284 tests (2 known issues, unchanged).
• SpectroKitParityTests run once, because a default changed: green, 83 tests.
• Red-first: the new tests check the default against the .off fit, which is the old default bit for bit (asserted). Before, Si read -8 to -57 % and Cu 0 to 0.42 at high rate; now they pass the bounds.
• Two independent numerics reviews before the commit. Fixed:
  • sigma_c scaled by the misfit;
  • the coincident-line caveat (Ar/Ag under Al+Al);
  • the self pair passing on the Poisson counts, now asserted;
  • the axis refinement cost (no sum-peak fits there);
  • several wording corrections.

What remains

• The docs/parity.md and docs/open-items.md rows, and the tools/edx-sim/README.md line for demo_pools.py, wait until the open PRs on those files land (each file is already in two open PRs).
• Not covered by a test:
  • the default with the axis refinement on (every new test locks the axis);
  • a pile-up spectrum with an incomplete element list, where the cross pairs could still pull on c.
• A real line under the self pair is flagged, not separated.
• Accuracy against published values on real data (a public Al-alloy EDS dataset or standard with a stated composition): next step, not in this PR.
• Auto ID (Auto ID leftovers after 0.11.0 #9): the proposeOnRefinedAxis path still lists Ar and Ag as possible candidates and does not give them the sum-peak question. That needs proposer files in other open PRs.

🤖 Generated with Claude Code