bb-plugin-variaq 0.5.0: VariaQ campaign and analysis support

.github/workflows/ci.yml

@@ -26,16 +26,16 @@ jobs:
       - run: npm pack --dry-run
 
   variaq-integration:
-    name: VariaQ 0.5.0 integration
+    name: VariaQ 0.6.0 integration
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
       - uses: actions/checkout@v4
       - uses: actions/checkout@v4
         with:
           repository: aaronphifer/variaq
-          # Immutable release tag for the verified VariaQ 0.5.x integration.
-          ref: v0.5.0
+          # Immutable release tag for the verified VariaQ 0.6.x integration.
+          ref: v0.6.0
           path: .variaq-source
       - uses: actions/setup-node@v4
         with:
@@ -52,7 +52,7 @@ jobs:
           "$RUNNER_TEMP/variaq-venv/bin/python" -m pip install --upgrade pip
           "$RUNNER_TEMP/variaq-venv/bin/python" -m pip install -e './.variaq-source[quantum]'
       - run: npm ci
-      - name: Run real local integration suite
+      - name: Run real local integration suite with bounded campaign smoke
         env:
           VARIAQ_TEST_PROJECT: ${{ github.workspace }}/.variaq-source
           VARIAQ_TEST_PYTHON: ${{ runner.temp }}/variaq-venv/bin/python

CHANGELOG.md

@@ -1,54 +1,51 @@
-## 0.4.0 (unreleased)
+## 0.5.0 (unreleased)
 
-- VariaQ 0.5.0 compatibility; `schema_version` `1` validation remains mandatory.
-- Assignment quantum support via QAOA and CUDA-Q (when VariaQ's `cudaq` extra is installed).
-- Subset Selection quantum support via QAOA and CUDA-Q.
-- Capability-driven quantum family discovery: solver `supported_families` are
-  consumed directly from `variaq capabilities --json`; no plugin-side hardcoded
-  solver/family matrix.
-- Generalized `variaq_compare_quantum`: works on any problem family supported by
-  the selected quantum solvers as reported by VariaQ capabilities.
-- Preserve VariaQ 0.5 BQM metadata, feasibility diagnostics (`feasible_sample_count`,
-  `infeasible_sample_count`), penalty metadata, expectation, lowered energy, and
-  timing fields without recalculation or flattening.
-- Graph Partition quantum attempts are rejected based on live VariaQ capabilities
-  (quantum support is not advertised in VariaQ 0.5.0).
-- Updated human-readable CLI formatting for solve/benchmark to show family, backend,
-  status, objective, expectation, and feasible/total samples where present.
-- Hosted CI updated to VariaQ `v0.5.0`; generic-family and Assignment/Subset quantum
-  coverage added. CUDA-Q remains optional in hosted CI.
+- VariaQ 0.6.0 compatibility; `schema_version` `1` validation remains mandatory.
+- Campaign planning (`variaq_campaign_plan`) and execution (`variaq_campaign_run`)
+  with VariaQ's max-run guard preserved.
+- Campaign list/show (`variaq_campaign_list`, `variaq_campaign_show`).
+- Analysis tools (`variaq_analyze_runs`, `variaq_analyze_campaign`) that return
+  VariaQ-derived groups, quality/feasibility/timing/resource summaries, and scaling
+  points without recalculating statistics in the plugin.
+- Report generation (`variaq_report_campaign`) for JSON/CSV/Markdown (and optional
+  plots when matplotlib is installed), with safe output path containment under the
+  configured report output directory.
+- Preserved source_run_ids, report IDs, null metrics, and mixed campaign status
+  summaries (success/failed/skipped/unavailable).
+- Hosted CI updated to VariaQ `v0.6.0` with bounded campaign plan/run,
+  analyze-campaign, and report-generation smoke coverage.
 - No external domain semantics, adapter execution, Ollama Fleet, Triagewall,
   physical QPU, UI, or web API added.
 
 # Changelog
 
 All notable changes to this project will be documented in this file.
 
-## [0.4.0] - Unreleased
+## 0.5.0 - Unreleased
 
-- Target VariaQ 0.5.x and its stable machine-readable CLI contract.
-- Recognize all four generic problem families: `maxcut`, `assignment`,
-  `subset-selection`, `graph-partition`.
-- Consume solver `supported_families` from `variaq capabilities --json`; let
-  VariaQ advertise which families each quantum solver supports.
-- Add quantum support for `assignment` and `subset-selection` through VariaQ 0.5.
-- Generalize `compare quantum` so it works for any family supported by the selected
-  quantum solvers, not only MaxCut.
-- Preserve VariaQ 0.5 BQM/feasibility/expectation metadata in solve, benchmark,
-  compare-quantum, and run records.
-- Reject Graph Partition quantum runs from VariaQ capabilities rather than a
-  hardcoded family list.
-- Update human-readable CLI formatting for multi-family results.
-- Update CI to verify against VariaQ `v0.5.0` and exercise quantum workflows for
-  Assignment and Subset Selection.
+- Target VariaQ 0.6.x and its stable machine-readable CLI contract.
+- Add campaign planning/execution/list/show tools and matching `bb variaq`
+  subcommands. Campaign execution preserves VariaQ's default max-run guard.
+- Add read-only analysis tools (`variaq_analyze_runs`, `variaq_analyze_campaign`)
+  that forward VariaQ-derived group summaries, quality, feasibility, timing,
+  resource, and scaling metrics without plugin-side recalculation.
+- Add `variaq_report_campaign` with JSON/CSV/Markdown output and optional plots.
+  Reports are written beneath a configured/safe report output directory with
+  path-traversal prevention.
+- Preserve provenance: `source_run_ids`, `report_id`, `campaign_id`,
+  `report_format_version`, `generated_at`, `variaq_version`.
+- Preserve mixed campaign status summaries and null/missing metrics.
+- Reject VariaQ 0.5.x and other non-0.6 series with a clear compatibility warning.
+- Update CI to verify against VariaQ `v0.6.0` and exercise bounded campaign
+  plan/run, analysis, and report generation.
 
 Physical-QPU execution, external domain adapter execution, and downstream
 project integrations are not part of this release.
 
-## [0.3.0] - Unreleased
+## 0.4.0 - Unreleased
 
-- VariaQ 0.4.x compatibility; `schema_version` `1` validation remains mandatory.
-- Family-aware problem generation for all VariaQ 0.4 generic families:
+- VariaQ 0.5.x compatibility; `schema_version` `1` validation remains mandatory.
+- Family-aware problem generation for all VariaQ 0.5 generic families:
   `maxcut`, `assignment`, `subset-selection`, `graph-partition`.
 - Capability model consumes VariaQ's per-solver `supported_families` from
   `variaq capabilities --json`; no plugin-side hardcoded solver/family matrix.
@@ -65,12 +62,12 @@ project integrations are not part of this release.
 - `runs list`, `run show`, and `run reproduce` are family-neutral.
 - Added `bb variaq status` human formatting for VariaQ capabilities.
 - Smoke script now exercises an `assignment` workflow in addition to MaxCut.
-- Updated hosted CI to VariaQ `v0.4.x` and added generic-family unit and
+- Updated hosted CI to VariaQ `v0.5.0` and added generic-family unit and
   integration coverage.
 - No external domain semantics, adapter execution, Ollama Fleet, Triagewall,
   physical QPU, UI, or web API added.
 
-## [0.2.0] - Unreleased
+## 0.2.0 - Unreleased
 
 - Target VariaQ 0.3.x and its stable machine-readable CLI contract.
 - Validate VariaQ `schema_version = "1"` envelopes; reject unknown future
@@ -92,7 +89,7 @@ project integrations are not part of this release.
 Physical-QPU execution and remote quantum-provider integration are not part
 of this release.
 
-## [0.1.0] - Unreleased
+## 0.1.0 - Unreleased
 
 - Initial thin integration with the standalone VariaQ CLI.
 - Eight experiment and persisted-run tools, plus status and version tools (ten

README.md

@@ -15,7 +15,10 @@ VariaQ schema-v1 CLI
   ↓
 VariaQ
   ├── classical
-  └── generic BQM-QAOA execution
+  ├── QAOA / CUDA-Q quantum simulators
+  ├── campaign planning, execution, and storage
+  ├── read-only analysis
+  └── report generation
 ```
 
 Starting with bb-plugin-variaq 0.2.0, the integration boundary is VariaQ's
@@ -25,21 +28,24 @@ or scrapes run IDs from prose.
 
 VariaQ remains fully usable without BB. The plugin is responsible only for BB
 tool/CLI registration, validated argument construction, bounded subprocess
-execution, and structured result forwarding.
+execution, safe temporary-file staging, report path containment, and structured
+result forwarding. The plugin does not implement campaign planning, analysis,
+statistics, report generation, solver logic, BQM logic, plotting, or provenance
+logic — VariaQ remains authoritative for all of these.
 
 ## Prerequisites
 
 - BB 0.43.x with Plugin SDK 0.4.104
 - Node.js 22.19 or a compatible version supported by BB
 - Linux-native Python 3.12
-- a separate VariaQ 0.5.x installation; 0.5.0 is the verified version
+- a separate VariaQ 0.6.x installation; 0.6.0 is the verified version
 
 Create a VariaQ environment separately from this repository:
 
 ```bash
 git clone https://github.com/aaronphifer/variaq.git
 cd variaq
-git checkout v0.5.0  # VariaQ 0.5.0
+git checkout v0.6.0  # VariaQ 0.6.0
 python3.12 -m venv .venv
 .venv/bin/python -m pip install -e '.[quantum]'
 ```
@@ -50,7 +56,7 @@ The `quantum` extra supplies the local Qiskit workflow. CUDA-Q is optional:
 .venv/bin/python -m pip install -e '.[quantum,cudaq]'
 ```
 
-VariaQ 0.5.0 constrains CUDA-Q to `>=0.16,<0.17`; do not independently
+VariaQ 0.6.0 constrains CUDA-Q to `>=0.16,<0.17`; do not independently
 upgrade it beyond VariaQ's supported range.
 
 ## Installation
@@ -81,6 +87,7 @@ Source defaults contain no machine-specific paths.
 | `projectDir` | empty | standalone VariaQ checkout; otherwise auto-discovered |
 | `dbPath` | empty | use VariaQ's `data/variaq.sqlite3` default |
 | `problemsDir` | empty | use VariaQ's `data/problems` default |
+| `reportOutputDir` | empty | safe default under the project workspace |
 | `timeoutMs` | `120000` | per-command timeout, allowed range 1,000–3,600,000 ms |
 
 Configured paths must be absolute. `pythonPath` must be an executable file and
@@ -92,21 +99,33 @@ bb plugin config variaq set pythonPath /path/to/variaq/.venv/bin/python
 bb plugin config variaq set projectDir /path/to/variaq
 bb plugin config variaq set dbPath /path/to/variaq/data/variaq.sqlite3
 bb plugin config variaq set problemsDir /path/to/variaq/data/problems
+bb plugin config variaq set reportOutputDir /path/to/variaq/reports
 bb plugin config variaq set timeoutMs 120000
 bb plugin reload variaq
 ```
 
 ## Agent tools
 
-The plugin registers ten BB tools:
+The plugin registers seventeen BB tools:
 
 - `variaq_status` and `variaq_version`
-- `variaq_problem_generate` and `variaq_problem_show`
+- `variaq_problem_generate`, `variaq_problem_show`, and `variaq_problem_import`
 - `variaq_solve`, `variaq_benchmark`, and `variaq_compare_quantum`
+- `variaq_campaign_plan`, `variaq_campaign_run`, `variaq_campaign_list`, and `variaq_campaign_show`
+- `variaq_analyze_runs` and `variaq_analyze_campaign`
+- `variaq_report_campaign`
 - `variaq_runs_list`, `variaq_run_show`, and `variaq_run_reproduce`
 
-The eight experiment/store tools are accompanied by the two status/version
-tools so agents can validate the local environment before running work.
+Tools are divided into read-only and execution/write categories:
+
+**Read-only:** `variaq_status`, `variaq_version`, `variaq_problem_show`,
+`variaq_runs_list`, `variaq_run_show`, `variaq_campaign_plan`,
+`variaq_campaign_list`, `variaq_campaign_show`, `variaq_analyze_runs`,
+`variaq_analyze_campaign`.
+
+**Execution / write:** `variaq_problem_generate`, `variaq_problem_import`,
+`variaq_solve`, `variaq_benchmark`, `variaq_compare_quantum`,
+`variaq_campaign_run`, `variaq_run_reproduce`, `variaq_report_campaign`.
 
 ## BB CLI and example workflow
 
@@ -125,29 +144,53 @@ bb variaq solve "$problem_id" --solver qaoa --seed 42 --json
 bb variaq benchmark "$problem_id" \
   --solvers exact,heuristic,qaoa --repeats 1 --seed 42 --json
 bb variaq compare-quantum "$problem_id" --p 1 --repeats 1 --json
+
+# Campaign workflow: plan first, then run, then analyze and report.
+campaign_json=$(cat <<'EOF'
+{
+  "campaign_format_version": "1",
+  "name": "maxcut-scaling",
+  "family": "maxcut",
+  "problem_sizes": [4, 6, 8],
+  "problem_seeds": [1, 2],
+  "solvers": ["exact", "heuristic"],
+  "repeats": 1,
+  "base_seed": 42,
+  "generator_parameters": {"edge_probability": 0.4},
+  "solver_config": {}
+}
+EOF
+)
+bb variaq campaign-plan "$campaign_json" --json
+bb variaq campaign-run "$campaign_json" --max-runs 100 --json
+bb variaq campaigns --limit 10 --json
+bb variaq campaign <campaign-id> --json
+bb variaq analyze-campaign <campaign-id> --group-by problem_id --group-by solver --scaling-x problem_size --json
+bb variaq report-campaign <campaign-id> --output-dir ./reports --formats json,csv,markdown --json
+
 bb variaq runs --limit 10 --json
 bb variaq run run-<uuid> --json
 bb variaq reproduce run-<uuid> --json
 ```
 
-All `solve`, `benchmark`, `compare-quantum`, and `reproduce` commands now
-return VariaQ's structured schema-v1 envelope directly. There is no prose
-parsing or run-ID scraping.
+All `solve`, `benchmark`, `compare-quantum`, `campaign-run`, and `reproduce`
+commands return VariaQ's structured schema-v1 envelope directly. There is no
+prose parsing or run-ID scraping.
 
 ## Version compatibility
 
 The verified combination is:
 
 ```text
-bb-plugin-variaq 0.4.0
-VariaQ           0.5.0
+bb-plugin-variaq 0.5.0
+VariaQ           0.6.0
 Plugin SDK       0.4.104
 BB host          0.43.x
 VariaQ schema    1
 ```
 
-bb-plugin-variaq 0.4.x is verified against VariaQ 0.5.x and schema version `1`.
-Patch releases within the 0.4 plugin series and 0.5 VariaQ series are accepted.
+bb-plugin-variaq 0.5.x is verified against VariaQ 0.6.x and schema version `1`.
+Patch releases within the 0.5 plugin series and 0.6 VariaQ series are accepted.
 Other VariaQ series or future schema versions are reported as unsupported with a
 clear compatibility error.
 
@@ -208,18 +251,39 @@ bb-plugin-variaq
 
 Project-specific adapters live outside this plugin.
 
+## Campaign, analysis, and reporting
+
+Starting with bb-plugin-variaq 0.5.0 and VariaQ 0.6.0:
+
+- **Campaign plan** (`variaq_campaign_plan` / `bb variaq campaign-plan`) is
+  non-executing and safe. It previews requested run counts, solver breakdown,
+  unavailable combinations, and max-run warnings.
+- **Campaign run** (`variaq_campaign_run` / `bb variaq campaign-run`) executes
+  solvers. It passes an explicit `maxRuns` value to VariaQ and only adds
+  `--override-max-runs` when the caller sets `overrideMaxRuns: true`. The plugin
+  does not bypass VariaQ's safety guard.
+- **Analysis** is read-only and returns VariaQ's `AnalysisResult` directly. The
+  plugin does not compute statistics, gaps, or scaling points.
+- **Reports** derive from stored runs and preserve `source_run_ids`,
+  `report_id`, `report_format_version`, `generated_at`, and `variaq_version`.
+  Report outputs are written beneath the configured `reportOutputDir` and cannot
+  escape that root (path traversal and absolute-outside-root paths are rejected).
+
 ## Limitations
 
-- VariaQ 0.5.0 provides local execution only; there is no physical-QPU path.
+- VariaQ 0.6.0 provides local execution only; there is no physical-QPU path.
 - There is no IBM Runtime/provider integration and no credential handling.
 - Generic CI does not require CUDA-Q, an NVIDIA GPU, a physical QPU, or remote
   services.
+- Campaign execution is local and single-threaded.
 - This plugin does not schedule work across Fleet and has no Triagewall
   integration.
 - Solver correctness belongs to VariaQ's own test suite; plugin tests cover the
   integration boundary.
 - A quantum run may return some infeasible samples; the plugin preserves
   VariaQ's feasibility diagnostics and does not treat them as a plugin error.
+- Optional matplotlib plots are only generated when VariaQ's reporting layer
+  detects matplotlib; they are not required for core functionality.
 
 ## Deterministic smoke test
 
@@ -259,9 +323,10 @@ npm run test:integration
 ```
 
 The normal GitHub Actions plugin job uses the deterministic fake CLI. A
-separate Linux integration job checks out the immutable VariaQ `v0.5.0` tag,
-installs `.[quantum]`, and exercises local classical/Qiskit workflows.
-CUDA-Q and GPU verification remain manual or suitable for a future self-hosted
+separate Linux integration job checks out the immutable VariaQ `v0.6.0` tag,
+installs `.[quantum]`, exercises local classical/Qiskit workflows, and runs a
+bounded campaign plan/run, analysis, and report-generation smoke. CUDA-Q, GPU,
+and matplotlib verification remain manual or suitable for a future self-hosted
 runner.
 
 See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution boundaries and