Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
11 changes: 11 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,14 @@ jobs:
top=$(grep -m1 -E '^## [0-9]+\.[0-9]+\.[0-9]+' CHANGELOG.md | sed -E 's/^## ([0-9]+\.[0-9]+\.[0-9]+).*/\1/')
tag=${GITHUB_REF_NAME#v}
[ "$top" = "$tag" ] || { echo "CHANGELOG top is $top, tag is $tag"; exit 1; }

private-refs:
name: No private references
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Scan tracked files
run: |
if git grep -n -I -E '_archive|CLAUDE\.md|memcap|macEM-suite|setup-python-refs' -- ':!.gitignore' ':!.github/workflows/ci.yml'; then
echo "Private references above; this repo is public."; exit 1
fi
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# published to GitHub Pages at https://mac4dstem.github.io/DiffractoKit/documentation/ on every tag.
# On a pull request the site is built and uploaded as an artifact only, so a broken catalog fails before the tag.
#
# No swift-docc-plugin: the kit takes no package dependencies (CLAUDE.md). `xcodebuild docbuild` builds the archives
# No swift-docc-plugin: the kit takes no package dependencies. `xcodebuild docbuild` builds the archives
# from the package's own schemes; `docc merge` (Xcode 16+) joins them. Needs Pages enabled with source "GitHub Actions"
# (repository Settings > Pages). Copied from CuboKit's pattern; the module list is read from the .docc catalogs.
name: Docs
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ The one numeric piece mac4DSTEM still carried that belongs here. No existing num
mac4DSTEM: build `radiiPerPosition` from its Bragg vectors (distance of each peak from the origin) and delete
`KnownCrystalQCalibration.estimate` (keep the app's `QCalibrationEstimate` façade or typealias to the kit's).

Not moved, by design: `ACOMCrystalSymmetry` / `OrientationMap` / `MetalACOMMatcher` (app, per its CLAUDE.md); the open phase-matching, ACOM-mirror and CIF-symmetry items are diagnose-first behaviour questions, not missing code.
Not moved, by design: `ACOMCrystalSymmetry` / `OrientationMap` / `MetalACOMMatcher` (app, by the app's own design); the open phase-matching, ACOM-mirror and CIF-symmetry items are diagnose-first behaviour questions, not missing code.

## 0.7.1 (2026-10-21)

Expand Down
2 changes: 1 addition & 1 deletion Tests/DiffractoOrientationTests/Fixtures/slice-4-pins.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion Tests/Fixtures/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
One row per committed fixture file under `Tests/*/Fixtures/`: size, the script under `tools/` that names it (the generator or its
pin script; a few scripts write a whole directory; a script found only by file-name prefix, tool-folder name or test module is marked so), and a note where the origin is not the kit's own pinned reference. 55 files.

Licence: every fixture is generated from this kit's own code and the pinned Python reference (see `CLAUDE.md`, Reference) and ships under the kit's GPL-3.0-or-later licence, except the rows marked *third party*. Nothing here is the owner's data: the committed text fixtures carry no owner path or name (checked by search), and real datasets are read from `$MACEM_DATA` and never copied into the repo.
Licence: every fixture is generated from this kit's own code and the pinned Python reference (named in the README) and ships under the kit's GPL-3.0-or-later licence, except the rows marked *third party*. Nothing here is the owner's data: the committed text fixtures carry no owner path or name (checked by search), and real datasets are read from `$MACEM_DATA` and never copied into the repo.

## Third party

Expand Down
2 changes: 1 addition & 1 deletion docs/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Float64 unless the function is Float32 by design (ACOM, ellipse pattern pixels).
| Pin note | `References/python/orix` runs pymatgen-core **2026.10.16** under a `pymatgen` 2026.9.24 metapackage; the fixtures record `pymatgen` 2026.9.24 and, except `cif-symmetry-pins.json` (generated 2026-10-21 on this install, byte-identical on regeneration), `pymatgen-core` 2026.9.23. Nothing here compares numbers, only time |
| 0.9.1 | The `symmetry_detection` and `cif_import` rows above are the 2026-10-21 measurement of 0.9.0. 0.9.1 replaced the closure check that took 95 % of them (see `docs/open-items.md`); measured in-process on the same `al_super2.cif` (machine load 3.6): `cif_import` typical median 67 ms (min 67, max 77), against 297 ms (one run at load 9) before the change and the table's 351 ms. Not yet re-run through `run.sh`; the nightly's row replaces these |

Re-run: `tools/benchmarks/gen_inputs.py` (References/python/orix Python), then `../../tools/memcap.sh --gb 8 -- tools/benchmarks/run.sh`
Re-run: `tools/benchmarks/gen_inputs.py` (References/python/orix Python), then `tools/benchmarks/run.sh`
for the kit and `tools/benchmarks/bench_python.py pymatgen|py4dstem` for Python, merged by `merge.py`. The raw lines are
`tools/benchmarks/results/2026-10-21.jsonl`.

Expand Down
2 changes: 1 addition & 1 deletion docs/open-items.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ A plain list, worst first. Each line: the item, the cause where it is known, wha
- **CIF symmetry.** Still unchecked: a group named by Hall symbol only, and a partial list whose count equals the primitive
group's order but whose operations are the wrong ones (a count cannot see that).
- **Python pins.** `References/python/orix` and `py4dstem` hold pymatgen-core 2026.10.16 under a `pymatgen` 2026.9.24 metapackage
(dist-info of both); `CLAUDE.md` and `docs/parity.md` say 2026.9.24, the older fixtures record pymatgen-core 2026.9.23 (a
(dist-info of both); the README and `docs/parity.md` say 2026.9.24, the older fixtures record pymatgen-core 2026.9.23 (a
previous venv). `cif-symmetry-pins.json` records `pymatgen` 2026.9.24 (a literal in `pins.py`) and was generated on this
install, so on pymatgen-core 2026.10.16: regenerating it on 2026-10-21 gave a byte-identical file. The other fixtures were not
regenerated; re-generate and re-pin them together at the next fixture change.
4 changes: 2 additions & 2 deletions docs/orientation-symmetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,12 +159,12 @@ not bit equality between different floating-point types. The measured Double-to-
From the worktree, with the existing pinned references (no installation or network access):

```sh
../../tools/memcap.sh -- ../../References/python/orix/bin/python -I -B tools/symmetry/pins.py
../../References/python/orix/bin/python -I -B tools/symmetry/pins.py
swift test -c release --jobs 4
```

The orix pins (`tools/orix/ipf-pins.py`, `tools/orix/disorientation-pins.py`) use the orix venv, provisioned by
macEM-suite `tools/setup-python-refs.sh orix` (orix 0.15.0, NumPy 2.5.4, SciPy 1.18.1, Python 3.12, freeze file in
a pinned-environment setup script (orix 0.15.0, NumPy 2.5.4, SciPy 1.18.1, Python 3.12, freeze file in
`References/python/orix/freeze.txt`); the direction-family pins use the py4DSTEM venv. `MACEM_SUITE_ROOT` can locate references from a differently placed checkout. `pins.py` invokes the pinned py4DSTEM
interpreter and compiles only a `git show` extraction of the app's symmetry classes at c809679c into this worktree's
`.build/symmetry-generator`. It does not build or modify the app. The JSON contains seed 20261016, package versions,
Expand Down
6 changes: 3 additions & 3 deletions docs/parity.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ function or a small group of them, not every method of the class.
yet. The "How checked" column says which; read it before relying on a row.
- **not held**: no kit symbol. "planned" names the place where it is registered or intended; it is not a promise of a date.

Numerics are held to the tolerance the Python package's own tests use; readers, I/O, indices and metadata are exact. Agreement with a package is agreement, not truth (`CLAUDE.md`); where the kit and a package differ by design the difference
Numerics are held to the tolerance the Python package's own tests use; readers, I/O, indices and metadata are exact. Agreement with a package is agreement, not truth; where the kit and a package differ by design the difference
is named. Counts are of rows.


Expand Down Expand Up @@ -84,8 +84,8 @@ and are counted apart.
| P22 | macTEM `RingOverlay.swift` (read-only; the app source is the oracle, no Python reference) transcribed into pure Python: 8-point ellipse polylines and the Newton point-to-ellipse distance (3600 samples, 60 iterations) for 4 ellipses x 7 points; closed forms for the circle and axis vertices | `tools/mactem-pieces/ring-overlay-pins.py` -> `Tests/DiffractoGeometryTests/Fixtures/ring-overlay-pins.json` | `RingOverlayTests` | polyline 1e-12; distance 1e-9 (absolute below 1); circle and vertex closed forms 1e-9 |
| P23 | macTEM `IndexingDisplay.swift` (read-only; the app source is the oracle, no Python reference) transcribed into pure Python: sigma_q = q tol/100, z, delta %, relative dd, margin, odds and the odds text on 6 accepted rings of the dyadic list g = 1, 1.5, 2, 3 (the decision is the kit's RingIndexing, P9) | `tools/mactem-pieces/indexing-display-pins.py` -> `Tests/DiffractoGeometryTests/Fixtures/indexing-display-pins.json` | `IndexingDisplayTests` | 1e-12 relative (margin and odds 1e-11); odds text and verdict names exact |
| P24 | macTEM `RingScale.swift` (read-only; the app source is the oracle): its header forms q = 2 sin(1/2 atan(R/L))/lambda, R = L tan(2 asin(lambda q/2)) written out in the test against the kit's `RingCalibration.q`/`radius` on a grid of 5 voltages x 4 spacings x 5 radii x 5 rings (500 cases); closed forms for the linear form and the file-versus-fit difference | none (closed forms in the test) | `RingScaleTests` | 1e-12 relative (the kit's measured 8.2e-16 for q, 3.3e-15 for R); linear form exact (one multiplication) |
| P25 | **(a) transcription of the app source at mac4DSTEM c809679c** (OrientationPlan, OrientationMatcher, the fundamental-zone samplers; NumPy, polar images and FFTs in float32 as the Swift): the beam directions, template images and spots, the experimental polar image and ring FFT, every template's score, the winner, runner-up, in-plane angle and orientation matrix, reduced Euler angles (m-3m), for m-3m Al (6 patterns), 6/mmm Mg (4) and 2/m (4), planted with the apps' excitation model, a 0.3 px jitter, spurious peaks and a Q calibration. **(b) py4DSTEM f050d207 run** (0.14.19, NumPy 2.5.0, SciPy 1.18.0): `Crystal.generate_diffraction_pattern` spots of 4 zone axes per crystal at lambda -> 0 and at 200 kV for 4 crystals (m-3m, 6/mmm, 2/m, 4/mmm); 18 planted orientations, py4DSTEM's own peaks, `orientation_plan` + `match_single_pattern` | `tools/acom-matcher/pins.py` with `app_route.py` (one process under `tools/memcap.sh`; sources' SHA-256 and versions in the fixture) -> `Tests/DiffractoOrientationTests/Fixtures/acom-matcher-pins.json` (540 kB; arrays base64) | `ACOMAppRouteTests` (`zoneAxesTemplatesAndSpotsEqualTheTranscription`, `matchesEqualTheTranscription`); `ACOMPy4DSTEMTests` (`spotsEqualPy4DSTEMsGeneratedPattern`, `plantedOrientationsAgreeWithPy4DSTEMsMatch`); `ACOMContractTests` | (a) beam directions 2e-14 (measured 2.3e-16); template images, polar image **bit exact** (gap 0); template spots 1e-6 (measured 1.2e-7, Float); ring FFTs 1e-6 (1.2e-7, vDSP vs NumPy); every template's score and the winner/runner-up 1e-6 (1.8e-7); orientation matrix 2e-6 (4.6e-7, the Float angle); in-plane angle 2e-6; template, mirror flag, Euler exact or 2e-6. Pinned choices are not near ties (margins > 1e-4 in the generator). (b) spots vs py4DSTEM 1e-6 (measured 5.1e-15 relative: r, azimuth and weight, both excitation models, 1808 spots); planted orientations: the kit's route with py4DSTEM's excitation model equals its NumPy transcription (matrix 1e-6) and lies 0.3-4.0 degrees from the truth and 0.4-3.9 degrees from py4DSTEM's answer for 17 of 18 patterns (py4DSTEM itself is 0.5-2.1 degrees from the truth); one 2/m pattern is matched to a different beam direction in the mirror pass (127 degrees), py4DSTEM finds it (1.5 degrees) Notes: the oracle environment is the CuboKit py4DSTEM venv (Python 3.12.4, NumPy 2.5.0, SciPy 1.18.0), not this repo's 3.14.7 / 2.5.4 / 1.18.1 pin set; the fixture records the versions it was made with. The kit requires at least `minimumPeakCount` (2) peaks for a polar image where the app only refuses an empty list (documented deviation; a one-peak image has no azimuthal structure to correlate). The pinned transcription includes patterns whose app-route answer is far from the planted truth (for example an Al pattern 36.7 degrees off at score 0.985, a monoclinic pattern 179 degrees off at 0.866 against a runner-up 0.861): these are the app's own outputs, pinned as such, and the near-ties are symmetry or pi ambiguities of the simplified route, not defects of the port. The reduced Euler angles are pinned to the app for m-3m only; the kit's reduction for 6/mmm and 2/m is checked against its own `CrystalOrientationSymmetry`, not against the app |
| P26 | orix 0.15.0 `DirectionColorKeyTSL(symmetry).direction2color` (the EDAX/TSL key as in MTEX) with `Vector3d.in_fundamental_sector` and `Symmetry.fundamental_sector`, all 11 Laue classes (orix's 2/m and -3m settings mapped to the kit's by a rotation, asserted as operator sets): 240 seeded signed random poles per class plus the axes, sector vertices, centres and points beside the vertices (2 777 poles) | `tools/orix/ipf-pins.py` (orix venv, macEM-suite `tools/setup-python-refs.sh orix`) -> `Tests/DiffractoOrientationTests/Fixtures/orix-ipf-pins.json` and the generated sector table `OrixIPFSectors.swift` | `OrixIPFKeyTests.orixKeyMatchesOrixForAllElevenLaueClasses`, `nativeKeysOfTheThreeOriginalClassesVersusOrix` | RGB 1e-5 absolute (orix's allclose; was 1e-7; measured 2.98e-8, the Float rounding); app native keys versus orix: m-3m 3.5e-6, 6/mmm 0.30, 2/m 1.0 (recorded, not widened) |
| P25 | **(a) transcription of the app source at mac4DSTEM c809679c** (OrientationPlan, OrientationMatcher, the fundamental-zone samplers; NumPy, polar images and FFTs in float32 as the Swift): the beam directions, template images and spots, the experimental polar image and ring FFT, every template's score, the winner, runner-up, in-plane angle and orientation matrix, reduced Euler angles (m-3m), for m-3m Al (6 patterns), 6/mmm Mg (4) and 2/m (4), planted with the apps' excitation model, a 0.3 px jitter, spurious peaks and a Q calibration. **(b) py4DSTEM f050d207 run** (0.14.19, NumPy 2.5.0, SciPy 1.18.0): `Crystal.generate_diffraction_pattern` spots of 4 zone axes per crystal at lambda -> 0 and at 200 kV for 4 crystals (m-3m, 6/mmm, 2/m, 4/mmm); 18 planted orientations, py4DSTEM's own peaks, `orientation_plan` + `match_single_pattern` | `tools/acom-matcher/pins.py` with `app_route.py` (one process under a memory cap; sources' SHA-256 and versions in the fixture) -> `Tests/DiffractoOrientationTests/Fixtures/acom-matcher-pins.json` (540 kB; arrays base64) | `ACOMAppRouteTests` (`zoneAxesTemplatesAndSpotsEqualTheTranscription`, `matchesEqualTheTranscription`); `ACOMPy4DSTEMTests` (`spotsEqualPy4DSTEMsGeneratedPattern`, `plantedOrientationsAgreeWithPy4DSTEMsMatch`); `ACOMContractTests` | (a) beam directions 2e-14 (measured 2.3e-16); template images, polar image **bit exact** (gap 0); template spots 1e-6 (measured 1.2e-7, Float); ring FFTs 1e-6 (1.2e-7, vDSP vs NumPy); every template's score and the winner/runner-up 1e-6 (1.8e-7); orientation matrix 2e-6 (4.6e-7, the Float angle); in-plane angle 2e-6; template, mirror flag, Euler exact or 2e-6. Pinned choices are not near ties (margins > 1e-4 in the generator). (b) spots vs py4DSTEM 1e-6 (measured 5.1e-15 relative: r, azimuth and weight, both excitation models, 1808 spots); planted orientations: the kit's route with py4DSTEM's excitation model equals its NumPy transcription (matrix 1e-6) and lies 0.3-4.0 degrees from the truth and 0.4-3.9 degrees from py4DSTEM's answer for 17 of 18 patterns (py4DSTEM itself is 0.5-2.1 degrees from the truth); one 2/m pattern is matched to a different beam direction in the mirror pass (127 degrees), py4DSTEM finds it (1.5 degrees) Notes: the oracle environment is the CuboKit py4DSTEM venv (Python 3.12.4, NumPy 2.5.0, SciPy 1.18.0), not this repo's 3.14.7 / 2.5.4 / 1.18.1 pin set; the fixture records the versions it was made with. The kit requires at least `minimumPeakCount` (2) peaks for a polar image where the app only refuses an empty list (documented deviation; a one-peak image has no azimuthal structure to correlate). The pinned transcription includes patterns whose app-route answer is far from the planted truth (for example an Al pattern 36.7 degrees off at score 0.985, a monoclinic pattern 179 degrees off at 0.866 against a runner-up 0.861): these are the app's own outputs, pinned as such, and the near-ties are symmetry or pi ambiguities of the simplified route, not defects of the port. The reduced Euler angles are pinned to the app for m-3m only; the kit's reduction for 6/mmm and 2/m is checked against its own `CrystalOrientationSymmetry`, not against the app |
| P26 | orix 0.15.0 `DirectionColorKeyTSL(symmetry).direction2color` (the EDAX/TSL key as in MTEX) with `Vector3d.in_fundamental_sector` and `Symmetry.fundamental_sector`, all 11 Laue classes (orix's 2/m and -3m settings mapped to the kit's by a rotation, asserted as operator sets): 240 seeded signed random poles per class plus the axes, sector vertices, centres and points beside the vertices (2 777 poles) | `tools/orix/ipf-pins.py` (the pinned orix venv) -> `Tests/DiffractoOrientationTests/Fixtures/orix-ipf-pins.json` and the generated sector table `OrixIPFSectors.swift` | `OrixIPFKeyTests.orixKeyMatchesOrixForAllElevenLaueClasses`, `nativeKeysOfTheThreeOriginalClassesVersusOrix` | RGB 1e-5 absolute (orix's allclose; was 1e-7; measured 2.98e-8, the Float rounding); app native keys versus orix: m-3m 3.5e-6, 6/mmm 0.30, 2/m 1.0 (recorded, not widened) |
| P27 | orix 0.15.0 `Orientation.reduce()` (single) and `Orientation.angle_with` / `Misorientation((sym, sym)).reduce()` (pairs), all 11 classes, versus brute force over the proper group (asserted in the generator): 160 single orientations (150 seeded random, 10 designed) and 150 pairs per class | `tools/orix/disorientation-pins.py` -> `orix-disorientation-pins.json` | `OrixDisorientationTests` | `reduce` Float angle atol 1e-8 + rtol 1e-5 of the angle (was 2e-7 rad; measured 1.19e-7); reduced matrix 1e-9 where the minimum is unique (measured 6.7e-16); `disorientation(from:to:)` angle atol 1e-8 + rtol 1e-5 (was 1e-9 rad; measured 1.4e-14). The kit's reduction needed no change |
| P36 | orix 0.15.0 `Misorientation((Gl, Gr))` (`equivalent`, `reduce`) for 12 group pairs and `Orientation.in_euler_fundamental_region` for the 11 Laue classes and -3m in both settings, groups built from the kit's own generators (`CrystalOrientationSymmetry.init`); beta'' in Al (the planted relation of `tools/beta-variants/planted.py`, 2/m to m-3m) | `tools/orix/misorientation-pins.py` -> `orix-misorientation-pins.json` | `OrixMisorientationTests` | equivalents as sets 1e-9 (measured 6e-16), smallest angle 1e-9 (4e-15) and orix's own angle 1e-4 where orix is minimal, reduced matrix 1e-9 where unique, Euler angles 1e-8 (2e-15); orix's `reduce` is not the minimum on 4 of 492 draws (see the DEVIATION in the Misorientation row) |
| P28 | py4DSTEM f050d207 `Crystal.symmetry_reduce_directions` run unchanged under its own `orientation_plan` ranges and symmetry_reduction (the P18 harness), all 11 classes, 2 387 seeded and designed matrices (7 161 columns) | `tools/symmetry/family-pins.py` (py4DSTEM venv) -> `py4dstem-family-pins.json` | `Py4DSTEMFamilyTests` | family entries 1e-7 absolute (was 1e-12; measured 0.0) on 6 990 columns; 171 columns within 1e-12 of a sector wall not compared (BLAS rounding decides the exact `>= 0` test) |
Expand Down
2 changes: 1 addition & 1 deletion tools/acom-matcher/pins.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""DiffractoKit ACOM matcher: the pin generator (docs/parity.md row P25).

tools/memcap.sh --gb 4 -- References/python/py4dstem/bin/python -I -B tools/acom-matcher/pins.py
References/python/py4dstem/bin/python -I -B tools/acom-matcher/pins.py

writes Tests/DiffractoOrientationTests/Fixtures/acom-matcher-pins.json. The py4DSTEM venv is the one of CuboKit (py4DSTEM
f050d207 = 0.14.19, NumPy, SciPy; versions are written into the fixture). Nothing here is generated from the new Swift.
Expand Down
Loading
Loading