feat: complete PDI fitting, review, calibration, and publication workflow

.github/workflows/ci.yml

@@ -0,0 +1,52 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - run: python -m pip install --upgrade pip
+      - run: python -m pip install -e '.[dev]'
+      - run: pytest
+        env:
+          MPLBACKEND: Agg
+          MPLCONFIGDIR: ${{ runner.temp }}/matplotlib
+
+  quality-and-build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+      - run: python -m pip install --upgrade pip
+      - run: python -m pip install -e '.[dev]'
+      - run: ruff check .
+      - run: ruff format --check .
+      - run: mypy
+      - run: python scripts/run_all_examples.py --output-dir "${RUNNER_TEMP}/xps-examples"
+        env:
+          MPLBACKEND: Agg
+          MPLCONFIGDIR: ${{ runner.temp }}/matplotlib
+      - run: python -m build
+      - name: Import the built wheel in a clean environment
+        run: |
+          python -m venv "${RUNNER_TEMP}/wheel-venv"
+          "${RUNNER_TEMP}/wheel-venv/bin/python" -m pip install dist/*.whl
+          "${RUNNER_TEMP}/wheel-venv/bin/python" -c "import xps_fitting; assert xps_fitting.__version__ == '0.3.0'"

.gitignore

@@ -16,4 +16,16 @@ __pycache__/
 .ipynb_checkpoints/
 
 # macOS
-.DS_Store
+.DS_Store
+
+# Legacy/generated data locations. Tracked raw spectra under data/raw are durable.
+raw_data/
+processed_data/
+processed_xps/
+outputs/
+figures/
+reports/
+graphify-out/
+build/
+dist/
+*.egg-info/

.graphifyignore

@@ -0,0 +1,5 @@
+outputs/
+graphify-out/
+build/
+dist/
+.venv/

.pre-commit-config.yaml

@@ -0,0 +1,19 @@
+repos:
+  - repo: local
+    hooks:
+      - id: ruff-check
+        name: Ruff lint
+        entry: ruff check --fix
+        language: system
+        types: [python]
+      - id: ruff-format
+        name: Ruff format
+        entry: ruff format
+        language: system
+        types: [python]
+      - id: mypy-public-api
+        name: mypy selected public API
+        entry: mypy
+        language: system
+        pass_filenames: false
+        always_run: true

.python-version

@@ -0,0 +1 @@
+3.13

CHANGELOG.md

@@ -0,0 +1,78 @@
+# Changelog
+
+## Unreleased
+
+## [0.3.0] - 2026-07-21
+
+- Added durable experimental candidate bundles with source/configuration hashes,
+  strict validation, and mandatory persistence before diagnostic plotting.
+- Added explicit immutable review versions and review records; structurally invalid,
+  synthetic, fixture-like, and raw-equals-fit bundles cannot be promoted.
+- Added sample manifests and persisted sample-wide calibration from the exact stored
+  C 1s centre, including reviewed raw Survey spectra and intensity-invariance checks.
+- Added plotting-only final PNG/PDF export from calibrated reviewed artifacts plus a
+  provenance JSON sidecar.
+- Added inspect, fit, review, calibrate, plot, validate, cleanup, and guided-wizard
+  CLI commands with conservative confirmation defaults.
+- Reorganised tracked acquisitions under `data/raw/`, fit hypotheses under
+  `configs/fits/`, and generated three raw-only sample manifests without changing
+  any VGD file bytes.
+- Added the beginner quick start, launcher, scientific workflow, review, calibration,
+  artifact/backup, schema, and troubleshooting documentation.
+
+- Refined detailed single-spectrum publication figures to 8 × 6 in with a complete
+  1.8 pt box, bold ticks, restrained markers/fills, a thinner total fit, additional
+  headroom, and compact framed legends without a background entry.
+- Standardised sentence-case chemical labels, including Aromatic C=C/C–C, and
+  layout-aware upper-left sample and upper-right core-level titles.
+- Added optional fitted binding-energy annotations with assignment colours,
+  one-decimal `eV` formatting, leaders, collision staggering, negligible/hidden
+  component controls, manual offsets, multipanel support, and recipe serialization.
+- Polished the publication, diagnostic, model-comparison, monochrome, PDI-series,
+  core-level-panel, and Cl 2p examples while retaining PNG/PDF-only export.
+- Added recipe-controlled y/top tick visibility, one-minor-tick spacing, optional
+  sample titles, axes-relative core labels, and 600 dpi output for the final C 1s plot.
+- Refined binding-energy annotation clearance against visible curves, other labels,
+  legends, and figure bounds while preserving fitted centres and component arrays.
+- Added a plotting-only PDI-H-COOH C 1s example that requires a reviewed Phase 1
+  export and writes only the named PNG and vector PDF.
+- Removed the synthetic publication smoke-test bundle and added strict experimental
+  provenance, stored raw/background, numerical-audit, and plot-immutability checks.
+- Preserved source-file, sample, and region metadata in new `FitResult` exports and
+  marked VGD acquisitions explicitly as experimental provenance.
+- Added non-mutating sample-wide binding-energy calibration for fitted results and
+  raw spectra, with one shared C 1s-derived offset, provenance, and double-shift guards.
+
+## [0.2.0] - 2026-07-21
+
+- Completed all eight deterministic, headless examples and added an isolated
+  all-examples runner.
+- Standardised figure output on PNG/PDF, centralised the 1.8 pt publication spine
+  contract, expanded semantic colours, and added publication size presets.
+- Added readable fit-bundle save/load APIs with numerically consistent reload.
+- Extended `xps-fit plot` to ordered multi-input bundle/curve sources with panel
+  labels, safe names, dry runs, collision checks, and concise ordinary errors.
+- Made the package version authoritative and recorded 0.2.0 in fitted-result
+  provenance.
+- Added Ruff, selected mypy checks, local pre-commit hooks, and Python 3.10–3.13 CI
+  with tests, examples, build, and clean-wheel import.
+- Exercised all tracked VGD spectra, real PDI-H-COOH C 1s candidate fits and Cl 2p
+  constraints, and raw PDI series/N 1s/O 1s plots with explicit scientific caveats.
+
+## Earlier Phase 1 foundation
+
+- Added validated CSV, XLSX, DataFrame, and optional VGD spectrum input.
+- Added area-normalised line shapes, backgrounds, chemistry constraints, staged
+  deterministic fitting, diagnostics, model comparison, and diagnostic plotting.
+- Added PDI-H-COOH C 1s and Cl 2p examples, export bundles, a stable `FitResult`
+  contract, and limited two-pass shared-shape fitting groundwork.
+
+## Earlier Phase 2 plotting foundation
+
+- Added scoped publication, diagnostic, monochrome, and presentation themes.
+- Added stable core-level and component palettes with alias handling and contrast checks.
+- Added single-spectrum, residual, multipanel, series, and candidate-model figures.
+- Added figure export plus validated JSON plotting recipes; version 0.2.0
+  standardises saved figures on PNG and PDF.
+- Added plotting-only CSV/XLSX/full-JSON reconstruction and the `xps-fit plot` CLI.
+- Added end-to-end numerical-integrity coverage and plotting documentation.

QUICKSTART.md

@@ -0,0 +1,201 @@
+# XPS workflow quick start
+
+This guide assumes you have never used Python, Git, or a command-line program.
+The workflow never changes the original VGD files.
+
+## 1. Install the tools on macOS
+
+Open **Terminal**: press Command-Space, type `Terminal`, then press Return.
+Install `uv` once by following <https://docs.astral.sh/uv/getting-started/installation/>.
+In Terminal, move into this repository (keep the quotation marks because the
+folder name contains a space):
+
+```bash
+cd "/Users/liangze/Desktop/squiddy tools/xps-fitting-workbench"
+uv sync --python 3.13
+uv run xps-fit doctor
+```
+
+You may instead double-click `Start_XPS_Workflow.command`. It checks `uv`, moves
+to the correct folder, installs the environment, and starts the guided wizard.
+
+## 2. Put raw files in the right folder
+
+Use one folder per sample under `data/raw/`:
+
+```text
+data/raw/PDI-H-COOH/
+  C1s Scan.VGD
+  N1s Scan.VGD
+  O1s Scan.VGD
+  Cl2p Scan.VGD
+  XPS Survey.VGD
+```
+
+Do not edit, rename during analysis, calibrate, or overwrite these files. The
+software hashes each file so a later change is detectable.
+
+## 3. Inspect the sample
+
+```bash
+uv run xps-fit inspect-sample --sample PDI-H-COOH
+```
+
+The report lists regions, point counts, energy ranges, acquisition metadata,
+SHA-256 hashes, missing regions, and existing candidate/reviewed/calibrated files.
+
+## 4. Fit the N 1s candidates
+
+```bash
+uv run xps-fit fit-region \
+  --sample PDI-H-COOH \
+  --region N1s
+```
+
+Each candidate is saved under `artifacts/candidates/` **before** diagnostic
+figures are created under `figures/diagnostic/`. Fitting does not approve a
+model and does not calibrate the data. For this sample, the N 1s candidates use
+one expected imide-N environment and compare linear and Shirley backgrounds.
+
+If you intentionally need to replace existing unreviewed candidates and figures,
+use `--overwrite-candidates --overwrite-figures`. Do not type `overwrite=True`;
+that is Python syntax, not a command-line option.
+
+## 5. Fit the O 1s candidates
+
+```bash
+uv run xps-fit fit-region \
+  --sample PDI-H-COOH \
+  --region O1s
+```
+
+The two- and three-component O 1s candidates test different assumptions about
+whether the carbonyl environments are resolved. Their structural area ratios are
+sensitivity checks, not automatic chemical truth. No adsorbate peak is added by
+default.
+
+## 6. Review N 1s and O 1s
+
+```bash
+uv run xps-fit review-region \
+  --sample PDI-H-COOH \
+  --region N1s
+
+uv run xps-fit review-region \
+  --sample PDI-H-COOH \
+  --region O1s
+```
+
+Read the listed centres, widths, areas, area fractions, residual statistics,
+AICc/BIC, bound hits, warnings, convergence information, correlations, and
+diagnostic paths. Open the diagnostic PNG or PDF. The menu lets you approve one
+candidate, reject all candidates, or cancel without changes. Approval creates a
+new version under `artifacts/reviewed/PDI-H-COOH/uncalibrated/`; it never changes
+the candidate. A lower AICc or BIC does not by itself prove a chemical assignment.
+
+## 7. Review Cl 2p and Survey
+
+```bash
+uv run xps-fit review-region \
+  --sample PDI-H-COOH \
+  --region Cl2p
+```
+
+For Cl 2p, check the constrained 2p3/2–2p1/2 doublet, separation, approximately
+2:1 area ratio, shared FWHM, and shared line shape.
+
+Survey has no peak model, so review it as a raw spectrum rather than inventing a
+fake fit:
+
+```bash
+uv run xps-fit review-spectrum --sample PDI-H-COOH --region Survey
+```
+
+## 8. Check whether calibration is ready
+
+```bash
+uv run xps-fit validate-sample \
+  --sample PDI-H-COOH
+```
+
+The report lists C1s, N1s, O1s, Cl2p, and Survey as `reviewed` or `missing` and
+prints the exact review command for anything missing. Calibration readiness must
+say `ready`. Do not use incomplete calibration for the final sample.
+
+## 9. Calibrate the complete sample
+
+Calibration uses the exact stored C 1s fitted centre, not a rounded plot label:
+
+```text
+energy shift = 284.8 eV - exact fitted reference centre
+```
+
+Run the command with the component key printed during review:
+
+```bash
+uv run xps-fit calibrate-sample \
+  --sample PDI-H-COOH \
+  --reference-region C1s \
+  --reference-component aromatic_C-C_C=C \
+  --target-energy 284.8
+```
+
+The command prints the exact centre, common shift, affected regions, missing
+regions, and the arrays that will not change. The default answer is No. If you
+continue, the identical shift moves C 1s, N 1s, O 1s, Cl 2p, Survey, and any
+other reviewed region for this sample in the same direction by the same amount.
+Intensities, background, components, residuals, widths, and areas do not change.
+The reviewed uncalibrated artifacts remain intact. Each compound needs its own
+calibration record.
+
+## 10. Generate all final figures
+
+```bash
+uv run xps-fit plot-sample \
+  --sample PDI-H-COOH \
+  --recipe configs/plots/pdi_publication.json
+```
+
+This creates individual Survey, C 1s, N 1s, O 1s, and Cl 2p figures plus the
+five-panel manuscript figure. Final PNG, PDF, and provenance JSON files go to
+`figures/final/PDI-H-COOH/`. Plotting reads only a reviewed calibrated bundle;
+it does not refit or reconstruct missing arrays. It refuses candidate,
+uncalibrated, synthetic, incomplete, or mixed-calibration inputs.
+
+To regenerate only N 1s, use:
+
+```bash
+uv run xps-fit plot-region \
+  --sample PDI-H-COOH \
+  --region N1s \
+  --recipe configs/plots/n1s_publication.json
+```
+
+Equivalent recipes are available for C 1s, O 1s, Cl 2p, and Survey.
+
+## 11. Validate and back up
+
+```bash
+uv run xps-fit validate-sample --sample PDI-H-COOH
+uv run xps-fit validate-calibration --sample PDI-H-COOH
+```
+
+Back up all of `data/raw/` and `artifacts/reviewed/`. Also keep `configs/`.
+Figures can be regenerated. Candidate artifacts are useful audit evidence but
+can be regenerated. `outputs/` and `figures/diagnostic/` are disposable after
+reviewed artifacts are safely backed up.
+
+Preview cleanup without deleting anything:
+
+```bash
+uv run xps-fit clean-generated --dry-run
+```
+
+See [troubleshooting](docs/troubleshooting.md) for common errors.
+
+If `xps-fit doctor` reports an unhealthy editable installation, run:
+
+```bash
+uv sync --python 3.13 --reinstall-package xps-fitting-workbench
+uv run xps-fit doctor
+```