Repository navigation
feat: auto ID explains candidates whose peak went to a detected sum peak - #61
Merged
Merged
Conversation
…aks as explained With the sum/escape model on, the unlisted-line check proposed on the bare spectrum and named the modelled pile-up and escape peaks as elements. SumEscapeResult.componentCounts (the fitted c S(A) + f E(A)) is now carried by the quantification and handed to ElementProposer.propose(explained:), which adds it to every fit as a fixed offset. nil (the default, and the model off) runs the proposer as before. Planted pools (tools/edx-sim proposer_sim, model on): 100 counts/px no longer names Ar, Ag, Pd; 300 counts/px no longer names Rh, Ar, In, Sn, Pd, Ag, Cd, Ru, Bi, K; a planted Fe escape with the free escape columns off no longer names La, Ce, Cr, Cs. A planted K K-alpha between two sum peaks is still named first. Model off: unchanged. Done: sum/escape peaks are explained counts for the proposer and the check. Next: docs rows for parity.md and open-items.md item 8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pair is significant The pooled quantification fitted one Gaussian per line and no sum (pile-up) peaks, exSpy's model. On the public synthetic AlMgSi demo (planted truth: the simulation recipe of sim_edx.py v2.0, not a measurement) that fit read noise-free Si K-alpha 15-18 % low on the Al-rich map and fixture pools and 57 % low on the highest-rate pool, held a planted Cu K-alpha at 0 at high rate, and had a reduced chi2 of 2.1-31 on the Poisson counts. The Al+Al sum peak (2.97 keV) and its neighbours pull the continuum segment above the Al K edge, and Si K-alpha with it. FitPolicy.sumPeaks (SumPeakRule, default .whenSignificant) now tests the strongest line's self pair alone with EDSSumEscapeFit; when its pile-up coefficient is 3 sigma above 0 (sigma scaled by sqrt(max(chi2_red, 1)) of that fit) the sum peaks of the four strongest lines are fitted and kept, otherwise the result is the old fit bit for bit. The self pair is the test because the cross pairs absorbed an unlisted Cu K-alpha (Ni K-alpha + O K-alpha) on a synthetic spectrum with no pile-up. The footer names the decision with its number, and names the alpha lines within one FWHM of the self pair (Ar K-alpha, Ag L-alpha under Al+Al): a real line there is read as pile-up. .off (FitPolicy.exspyEquivalent) is the old fit; a caller passing sumEscape keeps full control. The unlisted-line and fit-range refits repeat the reported decision; the proposer axis refinement runs without sum peaks. Noise-free Si K-alpha against planted: -0.0 to -4.0 % on five pools, -7.9 % at the highest rate; Cu K-alpha 92-97 % of planted at high rate (0-42 % before); reduced chi2 on the Poisson counts 0.70-1.08. The absorption-truth Si pin moves from -0.203 to -0.067 (+- 0.051), and the old pin stays on the .off fit. The Al K-alpha tail stays a detector calibration: measured once with TailEstimate on a high-count spectrum and typed, it brings Mg K-alpha from -22 to -32 % to within 5 % (matrix map -15 %). New fixture edx-demo-pools.json from tools/edx-sim/demo_pools.py. The golden scenarios pin the explicit off model; their numbers are unchanged, only the two new nil fields are added. Results stay validation none. Reviewed (numerics, two independent reviews): sigma scaling, the coincident-line caveat, the axis refinement cost and the wording fixed. BREAKING CHANGE: PooledQuantifier.run fits sum peaks by default when the data support them, so nets, at% and wt% change on spectra with pile-up. Migration: pass a policy with fit.sumPeaks = .off (or FitPolicy.exspyEquivalent), or sumEscape: SumEscapeModel(), to keep the old numbers. A stored QuantificationPolicy without the new field decodes as .off and its footer names the non-default policy. Done: sum peaks on by default in the pooled fit, with planted-truth tests on the demo pools. Next: the parity and open-items rows once the open PRs on those files land, a check of the default on real data with published values, and the sum-peak question for the auto ID possible candidates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The identification loop already gave the Al+Al sum peak (2.973 keV) its own column, but the element candidates that lost that peak to it (Ar K-alpha, Ag L-alpha, Pd L-alpha) stayed in the possible tier. Only proposed candidates got the sum-peak question. A candidate that clearly lost its peak to a sum column is now explained (ElementCandidate.explainedBySumPeak, ProposalResult.explainedBySumPeaks). Clearly means outside the ambiguity margin at the peak, no more total gain than the sum, and the sum column at or above its own L_D in the final fit. Explained candidates leave the possible list and the beside-K rule, and a note names them. A real line within the detector resolution of the sum cannot be told apart in one spectrum. It keeps its sum-peak conflict and the two-beam-current remedy. Red-first on the public AlMgSi demo pools (synthetic, planted truth from sim_edx.py v2.0): before, Ar, Ag and Pd were possible on L5, M_A and M_matrix. Now they are explained by Al+Al, and L7 (no Al) has no Al+Al explanation. Done: possible candidates get the sum-peak answer. Next: the golden baselines gain the new field (next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reason: ElementCandidate gained explainedBySumPeak, so every dumped candidate gains one line. Only lines are added: no number in eds-realistic-l-sim, eds-unlisted-sim or eds-proposer-sim changed. In eds-proposer-sim (beta'' region, planted sum peaks) nine unplanted candidates are now explained by the Al+Mg, Al+Si and Si+Si sums. Three notes lines name them. Done: baselines match. Next: PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main gained #59 (squashed, the base this branch was stacked on) and the continuum dead-layer term. Conflicts: - PooledQuantification.swift, QuantificationPolicy.swift: this branch does not change them; main's side kept (it is the branch side plus continuumDeadLayer). - the four golden baselines: main's files taken, then re-recorded with SPECTRO_GOLDEN_DIR. Against main the recording adds only the explainedBySumPeak = nil lines of this branch (35 in eds-realistic-l-sim, 60 in eds-unlisted-sim); demo-edx-tiny-dm4, eds-weakline-sim and eds-proposer-sim match exactly. No number moves. After the merge the diff against main is this branch's two commits only. Done: branch up to date with main, builds, gate green. Next: review and merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 31, 2026
…ns) into issue-9-possible-sum-question ElementProposer.swift conflicted on the notes block: keep both, the explained-as-pile-up notes and then the overlap-check note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paullo9
enabled auto-merge (squash)
November 1, 2026 12:42
…e-9-possible-sum-question The only conflicts were spectro goldens. They take the main side here; the next commit re-records them from the merged code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…default, no number moves Recorded from the merged code with SPECTRO_GOLDEN_DIR. Against main the only changes are this branch's own: the explainedBySumPeak field on proposal candidates and the three pile-up notes in the eds-proposer-sim region2 proposal. No numeric line moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Auto ID note for elements explained by a sum peak now reads "Explained as pile-up: ..." with no validation tag; the rest of the sentence is unchanged. DemoEDXAutoIDTests asserts the note carries no "(validation" text (red before, green after). Done: the pile-up note has no validation tag. Next: re-record the goldens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recorded from the code with SPECTRO_GOLDEN_DIR into a temporary folder and copied in. The only change is the three pile-up notes in the eds-proposer-sim region2 proposal; no numeric line and no csv hash moves. Done: goldens match the code. Next: CI on the branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paullo9
added a commit
that referenced
this pull request
Nov 1, 2026
Brings in the squash-merged EPQ line-weight table (#67), the sum-peak explanation for auto ID (#61) and the docs move (#76). One conflict, a doc comment on FitPolicy.init(from:): kept this branch wording, which names every key the decoder defaults (sumPeakSignificance included); main wording listed fewer. Goldens re-recorded with SPECTRO_GOLDEN_DIR after the merge: all six scenario files are byte-identical to the merged fixtures, so no number moved and no fixture changes in this merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Paul · project thread
Part of #9
Stacked on
issue-8-pooled-sum-peaks(#59). This PR's base is that branch; retarget tomainonce #59 merges. It is stacked becauseElementProposer.swiftis already in two open PRs (#59 and theissue-9-autoid-leftoversbranch). This is a third PR on that file, so expect a merge after those land. The touched regions differ: the loser loop, the candidate tiers and the notes.What changes
Auto ID (
proposeOnRefinedAxis, nothing listed) already gave the Al+Al sum peak (2.973 keV) its own column. The element candidates that lost that peak to it stayed in the possible tier, because only proposed candidates got the sum-peak question. On the Al-rich demo pools these were Ar K-alpha (15 to 27 L_D), Ag L-alpha (6 to 11 L_D) and Pd L-alpha.Now a candidate that clearly lost its peak to a sum column is explained:
ElementCandidate.explainedBySumPeakholds the sum label, andProposalResult.explainedBySumPeaksis the new tier.possible. They are not proposed, not a sum-peak question and not withheld.ProposalRules: an explained candidate no longer counts as a K peak in the beside-K rule (the peak is pile-up).What it cannot do: one spectrum cannot separate a real line from the sum when the two are within the detector resolution.
Additive public API: one new init parameter (default nil), one new property and one new computed tier. Apps that list
possibleshould also showexplainedBySumPeaks; otherwise those candidates drop from the list. The app change is not in this PR.Test evidence
The truth is planted on synthetic data, not a measurement: the public AlMgSi demo pools (mac4DSTEM
demo-edx,sim_edx.pyv2.0,edx-demo-pools.jsonfrom #59). No Ar, Ag or Pd is planted.Red first, before this change, on the Poisson counts of L5, M_A and M_matrix: Ar, Ag and Pd were in
possible, with reason "The peak at 2.97 keV went to Al+Al sum".After:
On L7 (pure Mg5Si6, no Al) there is no Al+Al column and nothing is explained by it.
New test file:
DemoEDXAutoIDTests(the three pools, a real Ar planted at 1000 and 4000 counts, and L7).Golden baselines: one line per dumped candidate (the new field) and three notes lines in
eds-proposer-sim. No number changed. These are in their own commit.Fast gate:
swift test -c release --parallel --num-workers 1 -j 1 --skip SpectroKitParityTestsis green: 287 SpectroEDX tests, 2 known issues (unchanged).Parity:
SpectroKitParityTestsran once, because the golden baselines moved. Green, 83 tests.One numerics review before the commit. It found no blocking bug. Fixed from it:
What remains for #9
issue-9-autoid-leftoversbranch.UnlistedLineCheckstill takes every candidate at or above L_D, explained or not. It runs on the quantification's own sum-peak decision (feat!: the pooled fit models sum peaks when the strongest line's self pair is significant #59), where the sum is a fixed explained part, so Ar and Ag are not candidates there. Not changed here.🤖 Generated with Claude Code