Release VariaQ v0.8.0 — scientific benchmarking and web hardening

.github/workflows/ci.yml

@@ -28,6 +28,42 @@ jobs:
       - name: Build distributions
         run: python -m build
 
+  base-wheel:
+    name: Base wheel without optional dependencies
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: python -m pip install build pytest
+      - run: python -m build
+      - run: python -m pip install dist/*.whl
+      - name: Verify unavailable backends and missing web hint
+        run: |
+          python - <<'PY'
+          import importlib.util
+          import json
+          import subprocess
+          import sys
+          for name in ("numpy", "qiskit", "cudaq", "fastapi"):
+              assert importlib.util.find_spec(name) is None, name
+          result = subprocess.run([sys.executable, "-m", "variaq", "capabilities", "--json"], capture_output=True, text=True, check=True, timeout=30)
+          envelope = json.loads(result.stdout)
+          assert envelope["schema_version"] == "1"
+          solvers = {item["name"]: item for item in envelope["data"]["solvers"]}
+          assert solvers["exact"]["available"]
+          for name in ("qaoa", "cudaq-cpu", "cudaq-gpu"):
+              assert not solvers[name]["installed"], name
+              assert not solvers[name]["available"], name
+          web = subprocess.run([sys.executable, "-m", "variaq", "web"], capture_output=True, text=True, timeout=30)
+          assert web.returncode == 2, web
+          assert "variaq[web]" in web.stdout + web.stderr, web
+          PY
+      - name: Test installed base wheel
+        run: python -m pytest tests/test_problems.py tests/test_experiments.py tests/test_identity.py tests/test_solvers.py tests/test_json_contract.py tests/test_capabilities.py tests/test_campaigns.py tests/test_paired_benchmarks.py tests/test_comparison_view.py tests/test_release_hardening.py
+
   test-windows:
     runs-on: windows-latest
     timeout-minutes: 15
@@ -40,7 +76,7 @@ jobs:
       - name: Install
         run: python -m pip install -e '.[dev,web]'
       - name: Test
-        run: pytest tests/test_problems.py tests/test_new_families.py tests/test_identity.py tests/test_capabilities.py tests/test_json_contract.py tests/test_cli.py tests/test_experiments.py tests/test_bqm.py tests/test_family_qaoa.py tests/test_campaigns.py tests/test_analysis.py tests/test_sense_and_historical.py tests/web
+        run: python -m pytest -q
       - name: Lint
         run: ruff check .
       - name: Formatting

.github/workflows/windows.yml

@@ -20,7 +20,7 @@ jobs:
       - name: Install
         run: python -m pip install -e '.[dev]'
       - name: Test core, storage, CLI JSON, capabilities, BQM, family baselines, structured output
-        run: pytest tests/test_problems.py tests/test_experiments.py tests/test_identity.py tests/test_cli.py tests/test_solvers.py tests/test_capabilities.py tests/test_json_contract.py tests/test_bqm.py tests/test_family_qaoa.py tests/test_new_families.py
+        run: pytest tests/test_problems.py tests/test_experiments.py tests/test_identity.py tests/test_cli.py tests/test_solvers.py tests/test_capabilities.py tests/test_json_contract.py tests/test_bqm.py tests/test_family_qaoa.py tests/test_new_families.py tests/test_campaigns.py tests/test_analysis.py tests/test_sense_and_historical.py tests/test_paired_benchmarks.py tests/test_comparison_view.py tests/test_release_hardening.py
       - name: Lint
         run: ruff check .
       - name: Formatting

.gitignore

@@ -31,6 +31,7 @@ data/artifacts/*
 # Legacy/root-level runtime paths used by older local invocations.
 /variaq.db
 /variaq.db-*
+*.web-worker.lock
 /problems/
 
 # Local credentials and environment files

CHANGELOG.md

@@ -3,6 +3,22 @@
 VariaQ follows semantic versioning while pre-1.0. Release dates will be added
 when a public release is created.
 
+## 0.8.0 — stable release (unreleased)
+
+- Added controlled scientific campaign matrices, authoritative per-instance quality,
+  conservative paired-solver measurements, variability, null reasons and expanded exports.
+- Added the responsive comparison workspace, reviewed experiment setup and local reproduction.
+- Persisted web task status in additive SQLite tables with an OS-held single worker lock.
+  Interrupted work retains completed attempts; duplicate web requests reconnect without retries.
+  Campaign attempts now persist before the next solver call, with recoverable membership provenance.
+- Removed solver execution from capability discovery; reads no longer select CUDA-Q targets.
+- Added first-backend-call timing with explicit overlap and lazy-initialization scope.
+  Cold GPU state remains unverified, and native calls cannot be safely cancelled.
+- Fixed canonical stored exact references, ambiguous duplicate campaign slots,
+  bounded web configuration, report symlink writes and cross-origin/forged-host requests.
+- Preserved experiment, campaign, report, CLI and API schema version 1 and existing history.
+- See [hardening and upgrade notes](docs/releases/v0.8.0-hardening.md).
+
 ## 0.7.0 — local web UI
 
 - Added an optional, local-first web interface (`pip install 'variaq[web]'`,

README.md

@@ -11,6 +11,14 @@ measure when quantum methods are useful, competitive, or impractical.
 
 **Classical · GPU · Quantum**
 
+## Reproducible scientific benchmarks (v0.8)
+
+The first v0.8 slice adds two bounded, opt-in MaxCut campaign matrices and
+per-instance paired analysis via `--compare paired_solvers`. Use the existing
+campaign and reporting commands; see [methodology and examples](docs/scientific-benchmarks.md).
+GPU availability and missing timing data are reported explicitly. Simulator
+results do not establish quantum advantage. This release is `0.8.0`; see the changelog and release notes.
+
 ## Why VariaQ exists
 
 Cross-framework comparisons are easy to distort accidentally. Different
@@ -34,7 +42,7 @@ physical QPU.
 
 ## Current capabilities
 
-VariaQ 0.7 supports domain-neutral optimization problem families:
+VariaQ 0.8 supports domain-neutral optimization problem families:
 
 - **MaxCut** — weighted graph partitioning into two sets.
 - **Assignment** — tasks/resources with scores, costs, capacities, demands, and
@@ -54,15 +62,15 @@ support:
   depends on the installed quantum backend and the binary-variable budget (see
   `variaq capabilities --json`).
 - **Graph Partitioning**: BQM lowering is implemented and tested, but quantum
-  support is experimental and not advertised for 0.7.0.
+  support is experimental in v0.8.
 
 Solvers:
 
 - exact enumeration with configurable state guards,
 - seeded multistart local-search heuristic,
 - Qiskit local statevector QAOA,
 - CUDA-Q `qpp-cpu` QAOA,
-- CUDA-Q `nvidia` and `nvidia-fp64` adapters when compatible hardware exists.
+- CUDA-Q `nvidia` adapters with explicit FP32/FP64 precision when compatible hardware exists.
 
 Cross-cutting features:
 
@@ -287,3 +295,10 @@ failure-handling, and remote-execution safety validation where applicable.
 ## License
 
 VariaQ is licensed under the [Apache License 2.0](LICENSE).
+
+VariaQ v0.8 includes a guided experiment workflow
+and scientific comparison workspace. See [benchmarking UI](docs/benchmarking-ui.md)
+for evidence selection, timing scopes, exports, reproduction, and current limits.
+
+See [hardening and upgrade notes](docs/releases/v0.8.0-hardening.md) for
+recovery, compatibility, security boundaries and current verification.

ROADMAP.md

@@ -83,9 +83,46 @@ These are directions, not implemented features or delivery commitments.
 - analysis-visualization charts driven by structured `AnalysisResult` data;
 - safe report downloads confined to known report artifacts.
 
-## v0.8 and later — later exploration
-
-Future candidates after 0.7 (none committed): richer interactive charts on the
+## v0.8 — reproducible scientific benchmarking (0.8.0)
+
+Shipped development slice:
+
+- small opt-in MaxCut matrices over sizes, graph seeds, solver repeats, and local
+  classical/Qiskit/CUDA-Q backends with existing resource guards;
+- authoritative per-instance quality and exact-reference gaps, feasibility,
+  conservative repeat pairing, source-run provenance, and repeat variability;
+- complete-call wall time and available compute/setup/warmup layers, with explicit
+  null/reason metadata for missing measurements and unverified cold starts;
+- separate numerical backend agreement from speed and sampled quality;
+- additive schema-v1 JSON, CSV, and Markdown reporting, documented methodology,
+  and dependency-free regression coverage.
+
+Web interface development slice:
+
+- cohesive dark-first responsive dashboard and navigation;
+- guided experiment recipes, capability-aware solver selection, advanced QAOA
+  configuration, plan invalidation, and local task recovery messaging;
+- bounded comparison workspace over authoritative Python instance/repeat data,
+  explicit pairing, null reasons, timing scopes, per-configuration scaling,
+  and selected JSON/CSV/Markdown report exports;
+- canonical run measurements, configuration/environment provenance, and local
+  reproduction with saved lineage;
+- keyboard-accessible scientific explanations and alternative chart data tables.
+
+Durable SQLite task snapshots and recovery of interrupted web tasks are implemented.
+Duplicate web submissions reconnect to the original task, and campaign attempts are
+saved before the next solver call. CLI reruns remain explicit new executions.
+See [web benchmarking workflow](docs/benchmarking-ui.md) and
+[hardening notes](docs/releases/v0.8.0-hardening.md). Hard native-call cancellation
+remains deferred.
+
+Fresh-process GPU cold-start verification, trustworthy peak memory telemetry,
+confidence intervals, and interrupting hung native calls remain future design work.
+See [scientific benchmark methodology](docs/scientific-benchmarks.md).
+
+## After v0.8 — later exploration
+
+Future candidates (none committed): richer interactive charts on the
 existing API, campaign execution progress refinement, stronger classical
 baselines evaluated only where they add scientific value, Graph Partition
 quantum research, and optional authenticated LAN mode for trusted labs.

SECURITY.md

@@ -11,7 +11,7 @@ Ordinary non-sensitive defects may use a normal bug report.
 
 ## Current model
 
-VariaQ 0.2 executes and stores experiments locally. Normal tests require no
+VariaQ executes and stores experiments locally. Normal tests require no
 credentials and no network. Never commit API keys, IBM credentials, cloud
 tokens, private keys, or user experiment databases.
 
@@ -26,4 +26,25 @@ queue/execution/charge timing, remain disabled in normal tests, and never be an
 implicit fallback.
 
 A future interface may require both `--backend ibm:<backend>` and
-`--allow-qpu`. This functionality does not exist in VariaQ 0.2.
+`--allow-qpu`. This functionality does not exist in VariaQ.
+
+## Local web boundary (v0.8 candidate)
+
+The default listener binds to `127.0.0.1`. Default local mode validates Host
+headers and refuses cross-origin/browser cross-site writes. It has no authentication
+and is not an internet service. Explicit non-loopback mode retains the trusted-lab
+boundary, prints a warning, and permits arbitrary hostnames; it is not hardened for
+DNS rebinding, hostile LAN clients or public exposure. Local applications can invoke
+the JSON API without browser Origin headers.
+
+Requests validate bounded generator dimensions and solver budgets, in addition to
+the existing run-count and solver state/memory guards. They provide no shell, raw SQL
+or arbitrary filesystem selector. Imported problems are JSON data with canonical
+family validation, not executable adapters. SQLite, problem/report directories and
+the adjacent worker lock are trusted local files; do not share writable access with
+untrusted users. Do not rename/delete the worker lock while the server is alive.
+Report exporters refuse output symlinks, and downloads enforce known artifact names
+and root containment. These checks do not defend against a hostile local process
+racing filesystem operations. Native GPU calls have no safe in-process timeout or
+cancellation. Stop the process if it hangs, inspect preserved records, and explicitly
+plan subsequent work; no automatic retry is performed.

benchmarks/maxcut_v0.8_unwarmed.campaign.json

@@ -0,0 +1,73 @@
+{
+  "campaign_format_version": "1",
+  "name": "maxcut-v0.8-unwarmed",
+  "family": "maxcut",
+  "problem_sizes": [
+    4,
+    6,
+    8
+  ],
+  "problem_seeds": [
+    11,
+    42
+  ],
+  "solvers": [
+    "exact",
+    "heuristic",
+    "qaoa",
+    "cudaq-cpu",
+    "cudaq-gpu"
+  ],
+  "repeats": 3,
+  "base_seed": 42,
+  "solver_config": {
+    "exact": {
+      "parameters": {
+        "max_states": 256
+      }
+    },
+    "heuristic": {
+      "parameters": {
+        "restarts": 4,
+        "max_passes": 16
+      }
+    },
+    "qaoa": {
+      "parameters": {
+        "p": 1,
+        "optimizer_trials": 8,
+        "shots": 128,
+        "warmup": false
+      }
+    },
+    "cudaq-cpu": {
+      "parameters": {
+        "p": 1,
+        "optimizer_trials": 8,
+        "shots": 128,
+        "warmup": false,
+        "precision": "fp64",
+        "max_statevector_bytes": 1048576
+      }
+    },
+    "cudaq-gpu": {
+      "parameters": {
+        "p": 1,
+        "optimizer_trials": 8,
+        "shots": 128,
+        "warmup": false,
+        "precision": "fp64",
+        "max_statevector_bytes": 1048576
+      }
+    }
+  },
+  "generator_parameters": {
+    "edge_probability": 0.4
+  },
+  "tags": [
+    "scientific-benchmark",
+    "v0.8",
+    "unwarmed"
+  ],
+  "notes": "Opt-in bounded simulator benchmark. No quantum advantage claim. Unwarmed does not prove fresh GPU state."
+}

benchmarks/maxcut_v0.8_warmed.campaign.json

@@ -0,0 +1,73 @@
+{
+  "campaign_format_version": "1",
+  "name": "maxcut-v0.8-warmed",
+  "family": "maxcut",
+  "problem_sizes": [
+    4,
+    6,
+    8
+  ],
+  "problem_seeds": [
+    11,
+    42
+  ],
+  "solvers": [
+    "exact",
+    "heuristic",
+    "qaoa",
+    "cudaq-cpu",
+    "cudaq-gpu"
+  ],
+  "repeats": 3,
+  "base_seed": 42,
+  "solver_config": {
+    "exact": {
+      "parameters": {
+        "max_states": 256
+      }
+    },
+    "heuristic": {
+      "parameters": {
+        "restarts": 4,
+        "max_passes": 16
+      }
+    },
+    "qaoa": {
+      "parameters": {
+        "p": 1,
+        "optimizer_trials": 8,
+        "shots": 128,
+        "warmup": true
+      }
+    },
+    "cudaq-cpu": {
+      "parameters": {
+        "p": 1,
+        "optimizer_trials": 8,
+        "shots": 128,
+        "warmup": true,
+        "precision": "fp64",
+        "max_statevector_bytes": 1048576
+      }
+    },
+    "cudaq-gpu": {
+      "parameters": {
+        "p": 1,
+        "optimizer_trials": 8,
+        "shots": 128,
+        "warmup": true,
+        "precision": "fp64",
+        "max_statevector_bytes": 1048576
+      }
+    }
+  },
+  "generator_parameters": {
+    "edge_probability": 0.4
+  },
+  "tags": [
+    "scientific-benchmark",
+    "v0.8",
+    "warmed"
+  ],
+  "notes": "Opt-in bounded simulator benchmark. No quantum advantage claim. Unwarmed does not prove fresh GPU state."
+}