feat: add centralized reciprocal-space reconstruction (GUI, CLI, cluster batch) - #69
Merged
Conversation
- add a C++17 reciprocal-space kernel for deterministic HKL and Q mapping with adaptive pixel and exposure-footprint subdivision - propagate pixel variances through weighted voxel accumulation while preserving source-pixel correlations and recording marginal-variance limits - add parallel image loading, correction, tiling, native mapping, and byte-bounded in-memory accumulation - implement immutable Parquet map partitions, bounded external reduction, resumable manifests, checksums, and one-write-per-chunk HDF5 finalization - freeze detector calibration, crystal, UB, scan references, corrections, exclusions, masks, backgrounds, and repair data from authoritative orGUI state - add exact and fallback exposure-angle bounds for supported scan backends and serializable references for regular, sliced, interlaced, manual, simulated, and external-backend scans - add a reusable native pixel-repair plan with interpolation variance propagation for constant detector masks - add the reciprocal-space reconstruction dialog with grid coverage, editable intervals, file-size estimates, geometry-matched Jacobian steps, HDF5 settings, performance controls, preview, run, and resume workflows - add shared global HDF5 chunk settings and an optional reconstruction compression override using the central filter registry - add orGUI rsmap and orGUI-rsmap run, resume, and status commands that consume UI-prepared job descriptors - write standalone NeXus-style HDF5 results with axes, intensity, variance, weight, contributors, configuration, build metadata, and provenance - register completed files as external orGUI results without copying large reconstructed arrays into the active database - add focused numerical, variance, geometry, correction, determinism, storage, CLI, dialog, resume, and native-kernel regression tests - add mapping, alignment, storage, and reducer benchmarks - document the complete reconstruction workflow, every user and job parameter, scientific equations, storage architecture, CLI usage, and practical tuning
Add a minimal CLAUDE.md that imports AGENTS.md so Claude Code picks up the existing guidance tree. Extend the AGENTS.md files to cover code that had grown outside their scope: the top-level reconstruction pipeline, orgui/backend/, and the xrayutils cpp/ native kernels, plus refresh stale file/test lists (reconstruction.py, ReconstructionDialog.py, test_reconstruction* suite). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Owner
Author
|
ToDo: before merge: settings for normalization in the integration tab should be moved to the RS map dialog. Other option: Provide a normalization API for the backend, (deterministic or with variance). This would break the existing output of all integrations. The raw output would then only be available through reconstruction of the normalized outputs. |
Owner
Author
|
Normalization in the integration tab is moved to the RS map dialog. Normalization rework is now opened as issue #70 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
orgui/datautils/xrayutils/reconstruction.py+ C++ kernel) that maps detector images directly into HKL/Q grids without materializing a full 3D array in memory.ReconstructionDialog— a GUI workflow (Experiment / Output grids / Performance / Cluster / Job-and-output / Preview tabs) to configure, preview, prepare, run, and resume reconstruction jobs, plus a sharedHDF5SettingsDialogfor output chunking/compression.reconstruction_cluster.py) with parallel submap reduction, and areconstruction_cli.pyentry point for headless/scripted runs.doc/source/reciprocal_space_reconstruction.rst, wired into the docs toctree, plusdoc/AGENTS.mddescribing the changelog/doc-sync workflow and a new automated doc-sync GitHub Action.Docs / known gaps
QScanSelector's new "Normalize by exposure time" / "Monitor corrections" fields live in the shared options panel (used by all scan workflows) but are currently only consumed by the reconstruction pipeline, not documented anywhere, and not mentioned as reconstruction-only inimage_integration.rst. Recommend a follow-up doc note (or scoping the fields to the reconstruction dialog) before release.CHANGELOG.md [Unreleased]entry for this feature yet.