Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 21 additions & 34 deletions Package.swift
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
// swift-tools-version: 6.2
// SpectroKit — a spectroscopy library (a Swift port of exSpy), extracted from an application's source at 5ba712a0.
// SpectroKit: EDS and EELS spectroscopy for the TEM, a Swift port of exSpy.
//
// Library targets (SpectroCore the shared part, SpectroEDX first, SpectroEELS later):
// `SpectroCore` is what EDS and EELS share (the spectrum image, the energy axis, the least-squares and Poisson fits, the
// counting statistics; the Velox and DM4 file readers moved to FormatoKit at 0.4.0); `SpectroEDX` is the X-ray part (lines, the continuum and EDS fit
// models, the element proposer, the quantification method, Cliff-Lorimer, absorption, the k-factor sources) and depends on
// SpectroCore. SpectroCore never depends on SpectroEDX; a future SpectroEELS will depend on SpectroCore only. Swift 6
// language mode, nonisolated by default (no `.defaultIsolation`), as in a consumer's core library. The six EPQ / FFAST / Krause tables
// are SpectroEDX's `Bundle.module` resources (`QuantificationTables.bundled()`, bundle `SpectroKit_SpectroEDX`); test
// fixtures are read by `#filePath` and are not resources. What one module needs from the other is `package`, never `public`.
// `SpectroCore` is what EDS and EELS share (the spectrum image, the energy axis, the least-squares and Poisson fits, the counting
// statistics); `SpectroEDX` (X-ray lines, the EDS fit, the element proposer, quantification, absorption) and `SpectroEELS`
// (thickness, alignment, deconvolution, core-loss edges) each depend on SpectroCore and never on each other. What one module needs
// from another is `package`, never `public`. Swift 6 language mode, nonisolated by default. The EDX tables and the EELS GOS tables
// are `Bundle.module` resources; test fixtures are read by `#filePath` and are not resources.
//
// 0.4.0: the file readers (`DM4Experiment`, `VeloxEMDReader`, `HDF5Serial`) live in FormatoKit (a Swift port of RosettaSciIO,
// docs/consumers-0.4.0.md), a package dependency on a FormatoKit minor range (`0.4.0..<1.0.0` since 0.7.1, `0.4.0..<0.6.0` at 0.7.0 (0.3.x dropped: the Velox golden needs the 0.4.0 metadata fields), `0.3.0..<0.5.0` at 0.6.1; `.upToNextMinor` at 0.6.0, exact before) so a consumer's `swift build` resolves it alone. FormatoKit
// ships libhdf5 as binary xcframeworks (FormatoLibHDF5, FormatoLibSZ, FormatoLibAEC), so nothing here asks a consumer to supply one.
// The file readers are FormatoKit's (a Swift port of RosettaSciIO), on a minor range so a consumer's `swift build` resolves it alone;
// FormatoKit ships libhdf5 as binary xcframeworks, so nothing here asks a consumer to supply one.
import PackageDescription

let package = Package(
Expand All @@ -25,14 +21,13 @@ let package = Package(
],
dependencies: [
.package(url: "https://github.com/mac4DSTEM/FormatoKit", "0.4.0"..<"1.0.0"),
// SpectroEELS runs its Fourier deconvolutions on ImagoCore's exact-length FFT (task 16, decision D25); 0.6.1 is the first tag with it.
// SpectroEELS runs its Fourier deconvolutions on ImagoCore's exact-length FFT; 0.6.1 is the first tag with it.
.package(url: "https://github.com/mac4DSTEM/ImagoKit", "0.6.1"..<"1.0.0"),
],
targets: [
// Plain-C wrappers over the four Accelerate LAPACK routines `FitLinearAlgebra` calls. ACCELERATE_NEW_LAPACK is
// defined for this C target only (a `cSettings` define, which SwiftPM accepts in a versioned dependency); its public
// header includes no Accelerate. This replaces S2's `-Xcc -DACCELERATE_NEW_LAPACK` unsafe flag: no unsafe
// flags anywhere in this manifest.
// header includes no Accelerate, so the manifest needs no unsafe flag.
.target(
name: "CSpectroLAPACK",
cSettings: [.define("ACCELERATE_NEW_LAPACK")],
Expand All @@ -46,20 +41,17 @@ let package = Package(
.product(name: "FormatoDM", package: "FormatoKit"),
.product(name: "FormatoEMD", package: "FormatoKit"),
],
// No Debug `-O` (S2 carried the source application's unsafe `-O` flag for the Debug configuration). Measured in S3:
// Debug at -Onone is 20-50x slower on every hot path (936 MB Velox 24.5 s vs 1.05 s, proposer 956 s vs 19.8 s),
// and `@_optimize(speed)` does not help at -Onone, but a versioned dependency cannot carry unsafe flags. So the
// optimisation is the consumer's: tools/test.sh runs this package's tests with `-c release`.
// No Debug `-O`: a versioned dependency cannot carry unsafe flags. Debug at -Onone is 20-50x slower on the hot paths
// (a 936 MB Velox file 24.5 s against 1.05 s, the proposer 956 s against 19.8 s), so optimising is the consumer's
// choice; tools/test.sh runs the tests with `-c release`.
swiftSettings: [
.enableUpcomingFeature("NonisolatedNonsendingByDefault"),
.enableUpcomingFeature("InferIsolatedConformances"),
.enableUpcomingFeature("MemberImportVisibility"),
]
),
// EELS slice 1 (docs/eels-slice-1-registration.md): the t/λ thickness map, a port of exSpy 7185a4d1's log-ratio
// method. Depends on SpectroCore only (EnergyAxis's HyperSpy rounding, FitLinearAlgebra's least squares). Slice 2
// (docs/eels-slice-2-registration.md): the C K edge by the ratio method; the dft GOS of the C, N and O K shells
// (CC-BY-4.0, NOTICE) are its `Bundle.module` resources (bundle `SpectroKit_SpectroEELS`).
// EELS: thickness, alignment, deconvolution and core-loss edges, ported from exSpy 7185a4d1. The dft GOS tables (CC BY 4.0,
// NOTICE) are its `Bundle.module` resources (bundle `SpectroKit_SpectroEELS`).
.target(
name: "SpectroEELS",
dependencies: ["SpectroCore", .product(name: "ImagoCore", package: "ImagoKit")],
Expand All @@ -82,9 +74,7 @@ let package = Package(
.enableUpcomingFeature("MemberImportVisibility"),
]
),
// The 258 tests that moved from the application the library was extracted from, by name (its test partition of 2026-10-13). They
// stay in one target across the split (the names are the contract), so it depends on both modules. At 0.4.0 the 68 reader
// tests among them moved to FormatoKit with the readers; the 190 others stay here, by name.
// The tests that came with the extraction from mac4DSTEM, most of them on SpectroEDX (issue #32 splits them by module).
.testTarget(
name: "SpectroCoreTests",
dependencies: [
Expand All @@ -98,8 +88,7 @@ let package = Package(
.enableUpcomingFeature("MemberImportVisibility"),
]
),
// SpectroKit's own tests, written after the extraction (0.2.0 on). Since 0.4.0 only the rank-threshold test: the Velox
// metadata fields, the DM4 EELS reader and the quit barrier moved to FormatoKit with their readers.
// Small package-wide tests: the CI scripts, the rank threshold, the smoothing kernel.
.testTarget(
name: "SpectroKitTests",
dependencies: ["SpectroCore", "SpectroTestSupport"],
Expand All @@ -109,7 +98,7 @@ let package = Package(
.enableUpcomingFeature("MemberImportVisibility"),
]
),
// EELS slice 1's tests: pins against exSpy, truth against planted spectra (Fixtures/, tools/eels-thickness/).
// SpectroEELS: pins against exSpy, truth against planted spectra (Fixtures/, written by tools/).
.testTarget(
name: "SpectroEELSTests",
dependencies: ["SpectroEELS", "SpectroCore", "SpectroTestSupport"],
Expand All @@ -120,9 +109,7 @@ let package = Package(
.enableUpcomingFeature("MemberImportVisibility"),
]
),
// EDX stage gates (docs/edx-stage-gates-registration.md): counts, energy axis and peak identification on pins (exSpy,
// HyperSpy, RosettaSciIO) and planted truth (Fixtures/, tools/edx-stages/), and the Gate D diagnosis of the Mg auto-ID
// miss. Its own target (registration Q6), so SpectroCoreTests stays the moved names (190 since 0.4.0).
// SpectroEDX: counts, energy axis and peak identification on pins (exSpy, HyperSpy, RosettaSciIO) and planted truth (Fixtures/).
.testTarget(
name: "SpectroEDXTests",
dependencies: ["SpectroEDX", "SpectroCore", "SpectroTestSupport", .product(name: "FormatoEMD", package: "FormatoKit")],
Expand All @@ -145,8 +132,8 @@ let package = Package(
.enableUpcomingFeature("MemberImportVisibility"),
]
),
// The pinned comparisons: the source application's byte-for-byte golden baseline (G-golden, recorded at its 71929a01), the
// exSpy/HyperSpy/pyEELSMODEL pins and the minute-long EDX gate runs (PeakGate, Gate D). `swift test -c release --parallel
// The pinned comparisons: the byte-for-byte golden baseline, the exSpy, HyperSpy and pyEELSMODEL pins and the minute-long
// planted EDX runs. `swift test -c release --parallel
// --skip SpectroKitParityTests` is the per-session gate; the plain `swift test -c release --parallel` runs both.
.testTarget(
name: "SpectroKitParityTests",
Expand Down
2 changes: 1 addition & 1 deletion Sources/CSpectroLAPACK/include/CSpectroLAPACK.h
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// CSpectroLAPACK.h
// Plain-C wrappers over the Accelerate LAPACK routines SpectroCore calls: the four of `FitLinearAlgebra` (SpectroKit S3,
// replacing a `-DACCELERATE_NEW_LAPACK` unsafe compiler flag), `dgelsd_` (SciPy's least squares, for the Savitzky-Golay port) and `dgbsv_` (SciPy's
// interpolating spline, EELS slice 2). This header includes no Accelerate: Swift sees only `int` and `double *`, so the
// interpolating spline). This header includes no Accelerate: Swift sees only `int` and `double *`, so the
// library needs neither `import Accelerate` nor the `-DACCELERATE_NEW_LAPACK` unsafe flag. The define lives in
// lapack_shim.c, the only file that sees Accelerate's LAPACK declarations.
//
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// BandedLinearAlgebra.swift
// Role: A general banded linear solve on Accelerate's LAPACK `dgbsv_` (LU with partial pivoting), the routine SciPy 1.18.1's
// `make_interp_spline` calls for its collocation system (`interpolate/_bsplines.py:1898-1909`). For SpectroEELS's
// interpolating spline (EELS slice 2); SpectroEDX does not use it.
// interpolating spline; SpectroEDX does not use it.
//

internal import CSpectroLAPACK
Expand Down
8 changes: 4 additions & 4 deletions Sources/SpectroCore/Spectroscopy/Fit/LeastSquaresFit.swift
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//
// LeastSquaresFit.swift
// Role: Unweighted least squares with non-negative line areas: the NAMED DEFAULT of the
// spectrum fit (ADR 054, V5-Q1). Every result carries `LeastSquaresFit.label`.
// spectrum fit. Every result carries `LeastSquaresFit.label`.
//
// Lawson and Hanson, "Solving Least Squares Problems" (1974), Algorithm 23.10 (NNLS), with
// the passive-set subproblem solved by Accelerate's QR-based `dgelsy_` (see
Expand All @@ -14,14 +14,14 @@
// DEVIATION (eXSpy/hyperspy): `Model1D.fit` with its default "ls" loss is an UNBOUNDED
// Levenberg-Marquardt on the line areas (`ext_force_positive` acts only with `bounded=True`),
// and it stops short of the least-squares minimum: on EDS_TEM_FePt_nanoparticles it returns
// 2800.0213 / 14921.183 where the exact minimiser is 2800.2296 / 14920.805 (docs/archive/v5 fixture,
// 7e-5 and 3e-5 relative). Here the minimum is exact and areas cannot go negative.
// 2800.0213 / 14921.183 where the exact minimiser is 2800.2296 / 14920.805 (7e-5 and 3e-5
// relative). Here the minimum is exact and areas cannot go negative.
//

import Foundation

package nonisolated enum LeastSquaresFit {
/// Printed beside every least-squares result (pre-registration flag 4).
/// Printed beside every least-squares result.
package static let label = "least squares, unweighted"

package struct Solution: Sendable, Equatable {
Expand Down
4 changes: 2 additions & 2 deletions Sources/SpectroCore/Spectroscopy/Fit/PoissonMLFit.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// PoissonMLFit.swift
// Role: Poisson maximum likelihood on the same linear design as the least-squares fit, by
// iteratively reweighted least squares (Fisher scoring) with non-negative line areas. It sits
// under Expert (ADR 054 V5-Q1); the cap and tolerance are recorded in every result.
// under Expert; the cap and tolerance are recorded in every result.
//
// The model is mu = D theta + offset (identity link), the data y_i ~ Poisson(mu_i). The score is
// s_j = sum_i D_ij (y_i - mu_i) / mu_i and the expected information F_jk = sum_i D_ij D_ik / mu_i, so
Expand All @@ -18,7 +18,7 @@
// infinite weight or an infinite deviance. The number of clamped channels is reported.
//
// Convergence: relative decrease of D below `tolerance` or `maxIterations` reached.
// The cap and tolerance are chosen by the session (pre-registration: "decides alone, recorded in the method").
// The cap and tolerance are chosen by the session.
//

import Foundation
Expand Down
2 changes: 1 addition & 1 deletion Sources/SpectroCore/Spectroscopy/MapSmoothing.swift
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
// A count-conserving spatial filter for element maps (Velox 3.15 "pre-filter": Average or Gaussian, "counts are
// redistributed and the total is conserved", UM pp. 64-68). Here it is display only: the controller applies it to the signed
// net-count map before the display clamp, the raw map stays what the file gave, the export writes the raw map, and every
// tile names the kernel. Nothing a quantification reads passes through it (ADR 054: at% on pooled spectra only).
// tile names the kernel. Nothing a quantification reads passes through it: at% is computed on pooled spectra only.
//
// Conservation: a normalised kernel conserves the total wherever its whole window is inside the map; at the border the
// weights are renormalised over the in-bounds taps, so a flat map stays flat to the edge (no dark rim, the mean per pixel is
Expand Down
2 changes: 1 addition & 1 deletion Sources/SpectroCore/Spectroscopy/SpectroCancellation.swift
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//
// SpectroCancellation.swift
// The one cancellation contract of the long spectroscopy calls (ADR 064, amendment (2) §2): a `cancelled` closure with the
// The one cancellation contract of the long spectroscopy calls: a `cancelled` closure with the
// default `SpectroCancellation.task`, an optional `progress` closure (fraction 0…1), and `SpectroCancelled` thrown when the
// closure answers true — nothing partial is kept.
//
Expand Down
6 changes: 3 additions & 3 deletions Sources/SpectroCore/Spectroscopy/SpectrumImage.swift
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
// the Velox event store.
//
// Counts are integers and every sum here is an exact `UInt64` sum, so a window map
// from the sparse store and from the same data densified is identical bit for bit
// (pre-registration C2): the only floating-point step is the net-count formula in
// from the sparse store and from the same data densified is identical bit for bit:
// the only floating-point step is the net-count formula in
// `WindowIntensity`, which both paths share.
//
// Joining Velox (lane A, `VeloxEventStore`, not in this tree): it already has
Expand Down Expand Up @@ -40,7 +40,7 @@ public nonisolated protocol SpectrumImage: Sendable {
/// (outer index = range, inner = pixel, row-major). Ranges are half-open and may overlap.
func windowSums(_ ranges: [Range<Int>]) -> [[UInt64]]

/// The same sums, cancellable (ADR 064 amendment 2 §2): `cancelled` is asked once per block of pixels, never per value, and
/// The same sums, cancellable: `cancelled` is asked once per block of pixels, never per value, and
/// `SpectroCancelled` is thrown with nothing kept; `progress` is monotone in 0...1 and ends at 1. A conformer that does not
/// implement it gets the default below (one check before the plain call). The default `cancelled` of the concrete types
/// and of `ElementWindows` is `SpectroCancellation.task`; a protocol requirement cannot carry it, so a caller through
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ public nonisolated enum SpectrumImageOpener {
}

/// Opens a file of a spectrum kind. For `.other` it throws `notASpectrumImage`.
/// Cancellable (ADR 064 amendment 2 §2): `SpectroCancelled` at a step boundary with nothing kept; `progress` 0...1, ends at 1.
/// Cancellable: `SpectroCancelled` at a step boundary with nothing kept; `progress` 0...1, ends at 1.
public static func open(path: String, kind: SpectrumFileKind,
cancelled: @Sendable () -> Bool = SpectroCancellation.task,
progress: (@Sendable (Double) -> Void)? = nil) throws -> LoadedSpectrumImage {
Expand Down
6 changes: 3 additions & 3 deletions Sources/SpectroCore/Spectroscopy/SpectrumImageSource.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
// protocol carries dimensions, axis, metadata and the compute (decision of lane R, taken by R2).
//
// The metadata is the file's own, as read. Live and real times are stored as recorded and are NOT interpreted:
// their meaning differs by file (cumulative in one Velox file, per frame in another, 0 on the owner's:
// formats.md §2.2), so nothing computes with them.
// their meaning differs by file (cumulative in one Velox file, per frame in another,
// 0 in a third), so nothing computes with them.
//

import Foundation
Expand Down Expand Up @@ -46,7 +46,7 @@ public nonisolated struct SpectrumImageMetadata: Equatable, Sendable {
/// Energy channels per spectrum.
public var channelCount: Int
/// The energy axis as the file states it: E(i) = offset + i · dispersion. Refinement on the pooled spectrum
/// (ADR 054 item 4) is shown beside it, never written over it.
/// is shown beside it, never written over it.
public var energyOffsetEV: Double
public var energyDispersionEV: Double
/// Real-space pixel size, when the file carries one.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
//
// CountingInterval.swift
// Role: Poisson counting statistics for window counts: the Garwood exact
// interval at EVERY count (ADR 054 addendum: the "~20 counts" switch was a
// threshold and is dropped), and the net count N = G - s*B with
// interval at EVERY count (no switch to a normal approximation at some
// count), and the net count N = G - s*B with
// Var = G + s^2*B. Inputs are plain numbers (gross G, background B, scale
// s = signal-window / background-window channel ratio), so the layer does
// not depend on the spectrum-image types.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//
// LiveTimeNormalisation.swift
// Role: Put regions with different dead time on a common footing (ADR 054,
// V5-Q5). Route 1: per-pixel live time, when the file carries it, gives a
// Role: Put regions with different dead time on a common footing.
// Route 1: per-pixel live time, when the file carries it, gives a
// rate = net counts / summed live time. Route 2: no live time, so the net
// count of a line is expressed relative to the Al K-alpha net count of the
// same region (the internal reference). The result NAMES the route.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
// protocol that the fit lane conforms to, and the delta-method ratio
// variance it feeds.
//
// k enters a Cliff-Lorimer ratio as k_A/k_B (ADR 054 addendum, flag 5), i.e. as
// k enters a Cliff-Lorimer ratio as k_A/k_B, i.e. as
// sqrt(sigma_kA^2 + sigma_kB^2) for independent k-factors (20 % each gives ~28 %).
// In a RATIO OF RATIOS of two regions both ratios use the same k_A/k_B, so the
// k term cancels exactly and `ratioOfRatios` carries it as 0.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ func proposeElements(counts: [UInt64], axis: EnergyAxis, beamEnergyKeV: Double)
## Record the method

A ``QuantificationMethod`` holds every choice the number depends on: the estimator, the background, the k-factor source, the
absorption correction, the elements and their roles. Its defaults are the registered ones.
absorption correction, the elements and their roles. Its defaults are listed on each property.

```swift
func method(for elements: [String], beamEnergyKeV: Double?) -> QuantificationMethod {
Expand Down
Loading
Loading