diff --git a/README.md b/README.md index 78ee6ea..2412750 100644 --- a/README.md +++ b/README.md @@ -167,6 +167,6 @@ matches it. Tags are made by the maintainer; what each tag changes for a consume - Big-endian DM files: none exists to test; not detected. - Velox: `lazy=True` is not ported; hostile grids and bin counts were run on FormatoKit only, not refuted against RosettaSciIO. Deliberate deviations are listed in `docs/parity.md`. -- Performance: `docs/benchmarks.md`. DM, TIFF and the 2048² Velox image read 1.2 to 2x slower than RosettaSciIO at typical sizes (`docs/open-items.md`); MIB, MSA, MRC, EMPAD, HDF5 4D and the Velox spectrum-image decode are at parity or faster. +- Performance: `docs/benchmarks.md`. DM, TIFF and the 2048² Velox image read 1.2 to 2x slower than RosettaSciIO at typical sizes (`docs/parity.md`, known limits); MIB, MSA, MRC, EMPAD, HDF5 4D and the Velox spectrum-image decode are at parity or faster. - TIFF: OME and SubIFD series, generic pyramids, compression and the other vendor branches are refused, not read (tifffile's shaped series are read since 0.11.0). - Validation: none. Agreement with RosettaSciIO and recovery of planted files are what is claimed. diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 5026baf..101da8f 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -64,7 +64,7 @@ Reading it: "Python / kit" above 1 is the kit faster. The Velox SI row is the st Readers stay on the CPU: they are bound by file reads and memory copies, there is no arithmetic for a GPU to take over, and host-to-device copies of a freshly read array cost more than the read. Measured break-even: none exists, no Metal path is a work item for any reader. Threading and mmap: only the 4D loads above 100 MB could split a read over cores; at the typical size the 4D rows -are already within 1.1x of h5py and of RosettaSciIO, so the question belongs to the large rows, not measured yet (`docs/open-items.md`). +are already within 1.1x of h5py and of RosettaSciIO, so the question belongs to the large rows, not measured yet (`docs/parity.md`, known limits). | Function | Backend | Why | |---|---|---| @@ -73,6 +73,6 @@ are already within 1.1x of h5py and of RosettaSciIO, so the question belongs to ## Slower than RosettaSciIO at the typical size -Listed first in `docs/open-items.md` with the cause. Found and fixed before this table (diagnosed with `sample`, then measured): +Listed first in `docs/parity.md` (known limits) with the cause. Found and fixed before this table (diagnosed with `sample`, then measured): MIB 934 ms to 35 ms (an `ISO8601DateFormatter` built per frame header, 1024 times, plus a per-pixel `Data.append`); MSA 111 ms to 17 ms (per-line allocation in the first reader). diff --git a/docs/open-items.md b/docs/open-items.md deleted file mode 100644 index 75cc25c..0000000 --- a/docs/open-items.md +++ /dev/null @@ -1,18 +0,0 @@ -# Open items - -Listed first: functions slower than RosettaSciIO at their typical size (`docs/benchmarks.md`, 2026-10-21). Cause stated where measured. - -1. **DM4 image, 2048² f32: 1.9 ms against 1.0 ms (0.54x).** Same for TIFF 4096² u16 (4.3 against 2.0) and DM4 4D 32²x128² (3.3 against 2.1). Cause - not isolated: the shared lazy file array zero-fills a `Data` and then `pread`s into it (one extra pass over the bytes); a try with - uninitialised storage was slower (the copy into `Data` costs more than the fill saves), so it was reverted. Next: read straight into - a `Data`-backed allocation without either pass, then re-measure. -2. **Velox image, 2048² x5: 9.9 ms against 8.5 ms (0.86x), MRC 1024² x10 and the small DM rows** are within the run-to-run spread of the - read path in item 1. -3. **Large rows of the benchmark are nightly** (`bench.py run --large`): decide threading for 4D loads above 100 MB there. - -Not done from the task list (each is a change that needs its own pins): -- The GMS EDS geometry (stage tilt, multi-segment list) so GMS absorption stops being refused: needs a real file (a GMS EDS `.dm4` with a spectrum image). Velox live and real times of `/Data/SpectrumImage` are read since 0.11.0 (`VeloxEMDReader.spectrumImageMetadata()`, the EDS signal's metadata); they are unvalidated against a second file, and an SI group with several streams or a disabled segment is covered only by planted files. -- TIFF series still refused by name: SubIFD series, OME, generic pyramids; a shaped description in a second ImageDescription tag (`description1`). -- DM4 lazy 4D gather speed (the 4D row above is the eager load; the whole-scan Float32 tile case is the app's; it needs a gather row in the nightly benchmark before a cause can be named); TIFF vendor branches (the Zeiss SEM, TVIPS and JEOL SightX metadata mappings, their axes being read under `readWithVendorAxes`; Olympus SIS and Hamamatsu entirely), ImageJ "nm", FEI2 extended MRC header; unbounded allocation from file sizes in Velox (`markersPerFrame`, frame transformations, detector times). -- Then-list: HSPY/ZSPY, compressed TIFF, NCEM/Berkeley EMD, SER/EMI, Bruker BCF, other writers. -- `_parse_image_display` is not run on the single-spectrum path of a SpectrumStream file. (`Sample.elements` is mapped for single spectra since #10 and for spectrum images since #45.) diff --git a/docs/parity.md b/docs/parity.md index a06c16b..75c672d 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -329,3 +329,18 @@ run the suite again without the variable. The regenerated files are committed on ## Peek (0.10.0) `Peek` has no RosettaSciIO counterpart. Its dimensions, axes, data types and acquisition fields are the readers' own (compared with them, exact); its pixels are the file's, point sampled at `stride = ceil(longer side / maxPixels)` (pixel (i, j) = source (i * stride, j * stride)), so there is no tolerance: a planted gradient (1000 * row + column) pins the closed form at stride 1 and 4, and every format is compared with the same file read whole by its own reader and cut at the same stride (`Tests/FormatoPeekTests`). A stack or cube is entered at the centre of its leading axes; a spectrum is binned to the largest value per bin. Budget: the 1.66 GB Velox spectrum image peeks in 0.36 s with 22 MB resident. + +## Known limits and decisions + +Open work is in the [issues](https://github.com/mac4DSTEM/FormatoKit/issues), including the ones closed as *not planned* and +parked for later (read speed #4, GMS EDS geometry #5, more formats #11). Speed against RosettaSciIO is in `docs/benchmarks.md`. + +- **Read speed.** DM4 2048² f32, TIFF 4096² u16 and DM4 4D 32² x 128² read about 2x slower than RosettaSciIO. The shared lazy + file array zero-fills a `Data` and then `pread`s into it (one extra pass); uninitialised storage was slower and was + reverted. Velox 2048² x5, MRC 1024² x10 and the small DM rows are within the run-to-run spread. Large rows are nightly + (`bench.py run --large`). +- **GMS EDS geometry** (stage tilt, multi-segment list) is not read, so GMS absorption is refused; it needs a GMS EDS `.dm4` + with a spectrum image. Velox live and real times of `/Data/SpectrumImage` are read since 0.11.0; an SI group with several + streams or a disabled segment is covered only by planted files. +- **FEI2 extended MRC header** is not read. `_parse_image_display` is not run on the single-spectrum path of a SpectrumStream + file. diff --git a/docs/status.md b/docs/status.md index 3ddbd94..05de08b 100644 --- a/docs/status.md +++ b/docs/status.md @@ -3,6 +3,6 @@ FormatoKit 0.11.0 (2026-10-22), against RosettaSciIO 0.14.0 (38 plugins on disk: 2 present, 5 partial, 31 missing, `docs/parity.md`). Live: Gatan DM3/DM4 (scan pixel size on the spectrum images), TIFF and tifffile's shaped series (0.11.0), MRC, Merlin MIB (pinned byte for byte, including the app convention), EMPAD, MSA/EMSA, Velox EMD (images, spectrum images with live and real time and the optics, single spectra, acquisition metadata with the pixel size), generic HDF5 read, append and write sessions, the BraggVectors writer, `FormatoLoader.sniff/load` over them, and `Peek` (one decimated frame and the header facts of a file, for Quick Look). Timings against -RosettaSciIO: `docs/benchmarks.md`; what is open (the GMS EDS geometry needs a real file; DM and TIFF read speed): `docs/open-items.md`. +RosettaSciIO: `docs/benchmarks.md`; what is open: the GitHub issues (the GMS EDS geometry needs a real file; DM and TIFF read speed); known limits: `docs/parity.md`. Test split (2026-10-21): the comparisons with the pinned RosettaSciIO and Pint values (`PinsTests`, `SpectrumImagePinsTests`, `UnitsTests`, `FormatoTIFFPinsTests`) and the inherited golden baseline are the `FormatoKitParityTests` target (fixtures stay in `Tests/Formato*Tests/Fixtures`, found through `FormatoKitTestSupport`); `tools/test.sh --parallel --skip FormatoKitParityTests` (or `swift test -c release --parallel --skip FormatoKitParityTests`) is the per-session gate (380 tests), plain `tools/test.sh --parallel` the full suite (396 tests: 380 fast + 16 parity, counted with `swift test list` at 0.11.0). The HDF5 exit-barrier test needs `tools/test.sh` (it sets `FORMATO_HDF5_PATH`). Timings of 2026-10-21 (Mac, release, warm build): fast target 8.2 s, full suite 9.4 s through `tools/test.sh`.