Upgrade analytic and code-to-code verification framework

CHANGELOG.md

@@ -11,6 +11,9 @@ Target release: 0.16.0, the first versioned MC/DC-VVP release.
 
 ### Added
 
+- Add Kornreich k-eigenvalue slabs problem
+- Add the analytical neutron $k$-eigenvalue MMS one-group slab, from [@ilhamv]
+- Add the analytical neutron fixed-source MMS two-group slab, from [@ilhamv]
 - Add the analytical neutron $k$-eigenvalue suite with subcritical and supercritical SHEM-361 cases, analytical matrix-eigenvalue references, active-cycle convergence studies, and uncertainty plots, from [@ilhamv]
 - Add energy-dependent weight-window and time-census variants of the infinite homogeneous SHEM-361 problem, from [@ilhamv]
 - Add AZURV1 variants for basic variance-reduction techniques, analytical spatial weight windows, time censuses, and census-based tallies, from [@ilhamv]
@@ -19,6 +22,7 @@ Target release: 0.16.0, the first versioned MC/DC-VVP release.
 
 ### Changed
 
+- Distribute the analytical fixed-source slab cases across the $x$, $y$, and $z$ axes to exercise every Cartesian slab orientation, from [@ilhamv]
 - Migration to Maestro-based launch, from [@ilhamv]
 - Update analytical fixed-source cases for the simulation-owned MC/DC interface and unified material model, from [@ilhamv]
 - Standardize **suite** and **case** as the VVP repository's organizational terminology, from [@ilhamv]

README.md

@@ -15,21 +15,36 @@ Workflow orchestration is performed using [Maestro](https://github.com/llnl/maes
 ```text
 configs/               Shared platform, user, and launch configurations
 verification/          Verification suites and their cases
-results/metadata.yaml  Append-only launch and processing history
-results/<launch_id>/   Processed results for one recorded launch
+results/               Processed results organized by suite
 
 launch.py              Launch all enabled suites
-process.py             Process one recorded launch
+process.py             Process suites and collect their results
+cleanup.py             Remove generated outputs and processed results
 ```
 
-MC/DC-VVP uses **suite** and **case** as standard terms for its two organizational levels:
+MC/DC-VVP uses **suite**, **case**, and **task** as standard terms for its organizational hierarchy:
+
+```text
+suite
+└── case
+    └── task
+```
 
 - A **suite** is a self-contained collection of related VVP cases with a shared launch and processing workflow.
 - A **case** is one individual problem definition and its inputs, reference solution or data, and processing logic.
+- A **task** is one execution of a case at one sampling level, such as one `N_particle` or `N_active` value.
 
-The top-level workflow launches and processes enabled suites, while each suite workflow runs and processes its cases.
+A suite contains cases, and each case generates one or more tasks from the suite's `task.yaml` configuration.
+Maestro currently represents each case as one workflow step, and `run_case.py` executes that case's tasks sequentially.
+The hierarchy also defines restart behavior: an existing output skips its task, a case with all task outputs is omitted, and a suite with all cases complete does not launch.
 Every integrated suite provides a README that describes its layout, configuration, workflow, and cases.
 
+## Adding content
+
+To add a case, create its directory under the appropriate suite's `cases/`, implement the common files described in the suite README, and register its task sequence in the suite's `task.yaml`.
+Place data shared by multiple cases in the suite's `data/` directory when appropriate.
+A new suite should provide its own README, `launch.py`, `process.py`, and `cleanup.py`, then be registered in `configs/launch_config.py.template`.
+
 ## Configuration
 
 Create the local launch configuration:
@@ -40,6 +55,11 @@ cp configs/launch_config.py.template configs/launch_config.py
 
 Edit `configs/launch_config.py` to enable the desired suites and set their platform and launch options.
 Use `platform=None` for local execution or a name from `configs/platform_config.py` for HPC execution.
+For HPC execution, `N_node` sets the number of nodes and each node uses all available CPU cores.
+For HPC execution, a suite's base `walltime` in hours is scaled by each case's `walltime_factor` in that suite's `task.yaml`.
+The scaled value is rounded up to the scheduler's supported resolution, the platform maximum remains the final limit, and local execution ignores walltime settings.
+Cases with every expected output are skipped, while partially complete cases retain their existing outputs and run only the missing sampling levels.
+Run the top-level `python cleanup.py` before launching when the entire configured campaign should start fresh.
 
 For HPC execution, also create `configs/user_config.py` from its template and provide the account and optional queue, reservation, and Python paths for the target platform.
 
@@ -59,40 +79,45 @@ python launch.py --platform tuolumne
 
 The `--platform` option selects suites with a matching configured platform.
 
-After the latest launch has completed, process all suites submitted by that launch:
+Process registered suites and collect their results:
 
 ```bash
 python process.py
 ```
 
-Process a specific recorded launch by passing the launch ID printed by `launch.py` and stored in `results/metadata.yaml`:
+For each suite registered in `configs/launch_config.py`, the top-level processor invokes the suite processor when a Maestro run is available and then moves the generated `results/` directory under the same suite path in the top-level `results/` directory.
+An existing suite `results/` directory can still be collected when no Maestro run is present, and suites with neither are skipped.
+Within each suite, `convergence/` contains study-wide convergence figures and `comparison/` contains plots or animations from the largest-statistics result.
+Collecting a suite replaces that suite's existing top-level results.
+
+Remove generated case outputs and processed results from every registered suite:
 
 ```bash
-python process.py 20260817T120000123456Z
+python cleanup.py
 ```
 
-Each launch is processed into its own `results/<launch_id>/` directory, which contains a metadata snapshot and the suite result hierarchy.
-Reprocessing one launch replaces only that launch's subfolder and does not affect results from other launches.
+Cleanup removes Maestro run directories and retains reference data.
 
 ## Suites
 
 ### Analytical verification
 
-Analytical verification demonstrates the expected statistical convergence of MC/DC by comparing numerical solutions against analytical reference solutions as the sampling effort is increased.
+Analytical verification demonstrates the expected statistical convergence of MC/DC by comparing numerical solutions against analytical and semi-analytical reference solutions as the sampling effort is increased.
 
 | Physics | Suite | Description |
 | :------ | :---- | :---------- |
-| Neutron transport | [Fixed-source](verification/analytical/neutron/fixed_source/README.md) | Multigroup steady-state and transient fixed-source cases, including Reed's problem, AZURV1 variants, and infinite SHEM-361 benchmarks. |
-| Neutron transport | [$k$-eigenvalue](verification/analytical/neutron/k_eigenvalue/README.md) | Infinite homogeneous SHEM-361 criticality cases with analytical eigenvalue and energy-spectrum references. |
+| Neutron transport | [Fixed-source](verification/analytical/neutron/fixed_source/README.md) | Multigroup steady-state and transient cases, including a two-group manufactured solution, Reed's problem, AZURV1 variants, and infinite SHEM-361 benchmarks. |
+| Neutron transport | [k-eigenvalue](verification/analytical/neutron/k_eigenvalue/README.md) | Homogeneous and Kornreich-Parsons one-group slab benchmarks, plus infinite homogeneous SHEM-361 cases. |
 
-### Benchmark verification
+### Code-to-code verification
 
-Benchmark verification compares MC/DC against established reference Monte Carlo codes for cases without analytical solutions.
+Code-to-code verification assesses whether relative differences among independently implemented transport codes decrease at the expected statistical rate as their sampling effort increases.
+Convergence proportional to $N^{-1/2}$ supports that the participating codes are approaching the same solution at the expected Monte Carlo rate, although agreement alone cannot exclude shared bias.
+The arithmetic mean of all participating code estimates at the largest sampling level defines a fixed comparison reference for every level, allowing a case to include two or more codes without designating one as exact.
 
 | Physics | Suite | Description |
 | :------ | :---- | :---------- |
-| Neutron transport (multigroup) | Benchmark multigroup | Time-dependent benchmark cases, including the Kobayashi Dog-Leg and C5G7 transient benchmarks. |
-| Neutron transport (continuous energy) | [Benchmark continuous energy](verification/benchmark/neutron/continuous_energy/README.md) | Continuous-energy benchmark cases for representative reactor systems. |
+| Neutron transport | [Code-to-code](verification/code_to_code/neutron/README.md) | Time-dependent C5G7 and Kobayashi comparisons among participating codes. |
 
 ## Validation
 

cleanup.py

@@ -0,0 +1,65 @@
+"""Remove generated outputs and processed results from all registered VVP suites."""
+
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+# ======================================================================================
+# Bootstrap VVP imports
+# ======================================================================================
+
+REPO_DIR = Path(__file__).resolve().parent
+
+# Support running this script from any working directory without installing VVP.
+if str(REPO_DIR) not in sys.path:
+    sys.path.insert(0, str(REPO_DIR))
+
+
+# ======================================================================================
+# Load suite configuration
+# ======================================================================================
+
+from configs.launch_config import LAUNCH_CONFIG
+
+# ======================================================================================
+# Clean registered suites
+# ======================================================================================
+
+for suite in LAUNCH_CONFIG:
+    suite_dir = REPO_DIR / suite
+    cleaner = suite_dir / "cleanup.py"
+
+    if not suite_dir.is_dir():
+        raise FileNotFoundError(f"Suite directory not found: {suite_dir}")
+
+    if not cleaner.is_file():
+        print(f"Skip suite without cleanup.py: {suite}")
+        continue
+
+    print("=" * 80)
+    print(f"Cleaning suite: {suite}")
+    print("=" * 80)
+
+    subprocess.run(
+        [sys.executable, str(cleaner)],
+        cwd=suite_dir,
+        check=True,
+    )
+
+
+# ======================================================================================
+# Remove collected results
+# ======================================================================================
+
+results_dir = REPO_DIR / "results"
+if results_dir.is_dir():
+    shutil.rmtree(results_dir)
+
+
+# ======================================================================================
+# Summary
+# ======================================================================================
+
+print()
+print("Cleanup complete.")

configs/launch_config.py.template

@@ -2,15 +2,19 @@ LAUNCH_CONFIG = {
     "verification/analytical/neutron/fixed_source": {
         "enabled": False,
         "platform": None,
-        "mpi": False,
+        "N_node": 1,
         "walltime": None,
-        "rewrite": False,
     },
     "verification/analytical/neutron/k_eigenvalue": {
         "enabled": False,
         "platform": None,
-        "mpi": False,
+        "N_node": 1,
+        "walltime": None,
+    },
+    "verification/code_to_code/neutron": {
+        "enabled": False,
+        "platform": None,
+        "N_node": 1,
         "walltime": None,
-        "rewrite": False,
     },
 }

configs/platform_config.py

@@ -12,7 +12,8 @@
         "gpus_per_node": 0,
         "max_nodes": 128,
         "max_walltime_hours": 24,
-        "walltime_format": "{hours}:00:00",
+        "walltime_resolution_seconds": 1,
+        "walltime_format": "{hours}:{minutes:02d}:{seconds:02d}",
     },
     "lassen": {
         "host": "lassen",
@@ -23,7 +24,8 @@
         "gpus_per_node": 4,
         "max_nodes": 128,  # actual system limit: 256
         "max_walltime_hours": 24,  # actual system limit may be lower
-        "walltime_format": "{hours}:00",
+        "walltime_resolution_seconds": 60,
+        "walltime_format": "{hours}:{minutes:02d}",
     },
     "tuolumne": {
         "host": "tuolumne",
@@ -34,6 +36,7 @@
         "gpus_per_node": 4,
         "max_nodes": 128,
         "max_walltime_hours": 24,
-        "walltime_format": "{hours}:00:00",
+        "walltime_resolution_seconds": 1,
+        "walltime_format": "{hours}:{minutes:02d}:{seconds:02d}",
     },
 }

configs/util.py

@@ -0,0 +1,58 @@
+"""Shared helpers for VVP configuration and launch processing."""
+
+import math
+from pathlib import Path
+
+
+def case_outputs_complete(case_dir, sample_counts):
+    """Return whether every expected sample output exists for a case."""
+    case_dir = Path(case_dir)
+    return all(
+        (case_dir / f"output_{int(count)}.h5").is_file() for count in sample_counts
+    )
+
+
+def get_case_walltime(task, platform, walltime_base=None):
+    """Scale the launch walltime for one case and apply the platform limit."""
+    walltime_factor = task.get("walltime_factor", 1.0)
+
+    # Validate the optional launch-wide base duration.
+    if walltime_base is not None and (
+        isinstance(walltime_base, bool)
+        or not isinstance(walltime_base, (int, float))
+        or not math.isfinite(walltime_base)
+        or walltime_base <= 0.0
+    ):
+        raise ValueError("Launch walltime must be greater than zero.")
+
+    # Validate the multiplier supplied by the individual case.
+    if (
+        isinstance(walltime_factor, bool)
+        or not isinstance(walltime_factor, (int, float))
+        or not math.isfinite(walltime_factor)
+        or walltime_factor <= 0.0
+    ):
+        raise ValueError("Case walltime_factor must be greater than zero.")
+
+    # Use the platform limit as the base when the launch does not specify one.
+    if walltime_base is None:
+        walltime_base = platform["max_walltime_hours"]
+
+    # Round up to the scheduler resolution, then enforce the platform limit.
+    resolution = platform["walltime_resolution_seconds"]
+    max_seconds = (
+        math.floor(platform["max_walltime_hours"] * 3600.0 / resolution) * resolution
+    )
+    walltime_seconds = (
+        math.ceil(walltime_base * walltime_factor * 3600.0 / resolution) * resolution
+    )
+    walltime_seconds = min(walltime_seconds, max_seconds)
+
+    # Convert seconds into the scheduler-specific walltime syntax.
+    hours, remainder = divmod(walltime_seconds, 3600)
+    minutes, seconds = divmod(remainder, 60)
+    return platform["walltime_format"].format(
+        hours=hours,
+        minutes=minutes,
+        seconds=seconds,
+    )