Add batched GPU simulator and release 0.2.0

.github/workflows/publish.yml

@@ -0,0 +1,55 @@
+name: Publish to PyPI
+
+# Single publication path. PyPI Trusted Publishing is bound to this file name
+# (publish.yml) and the "pypi" environment. Tests must pass and the release tag
+# must match the pyproject version before anything is uploaded.
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    uses: ./.github/workflows/tests.yml
+
+  build:
+    needs: test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Check release tag matches package version
+        if: github.event_name == 'release'
+        run: |
+          version=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml', 'rb'))['project']['version'])")
+          if [ "${GITHUB_REF_NAME#v}" != "$version" ]; then
+            echo "::error::Release tag $GITHUB_REF_NAME does not match pyproject version $version"
+            exit 1
+          fi
+      - name: Build and check distributions
+        run: |
+          python -m pip install --upgrade build twine
+          python -m build
+          twine check --strict dist/*
+      - uses: actions/upload-artifact@v4
+        with:
+          name: python-package-distributions
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: python-package-distributions
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/tests.yml

@@ -0,0 +1,26 @@
+name: Tests
+
+on:
+  workflow_call:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[test]"
+      - name: Test
+        run: pytest tests/ -v

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.2.0 — 2026-10-27
+
+- Added `westquant_cudaq.batched`: batched state-vector simulation of many small
+  circuits (NumPy on CPU, CuPy on GPU) with Pauli-trajectory noise, readout
+  error, shots and Parquet output with provenance. Validated against Qiskit
+  `Statevector` and an exact density-matrix reference.
+- Added the `wq-batch` command (`run`, `plan`, `bench`) and Slurm scripts for
+  A100 clusters in `scripts/hpc/`. See `docs/BATCHED.md`.
+- New optional extras `batched` and `gpu`. `import westquant_cudaq` stays
+  dependency-free; `westquant_cudaq.batched` is imported lazily.
+- The module became a package (`westquant_cudaq/`); the public 0.1.0 API is
+  unchanged and its tests still pass.
+- CI: tests workflow and a test-gated PyPI Trusted Publishing workflow.
+
 ## 0.1.0 — 2026-09-29
 
 - Added framework-neutral workload profiles and representation candidates.

CITATION.cff

@@ -9,5 +9,5 @@ authors:
 repository-code: "https://github.com/WestQuantOpen/westquant-cudaq"
 url: "https://github.com/WestQuantOpen/westquant-cudaq"
 license: Apache-2.0
-version: "0.1.0"
-date-released: "2026-09-29"
+version: "0.2.0"
+date-released: "2026-10-27"

README.md

@@ -106,6 +106,19 @@ WestQuant never executes an unverified representation candidate. Constraints for
 
 `execute_cudaq` imports CUDA-Q lazily and supports kernel execution, sampling, and observation. No CUDA dependency is imported when using only planning and passport APIs.
 
+## Batched GPU simulation for data generation (new in 0.2.0)
+
+`westquant_cudaq.batched` simulates thousands of small circuits per GPU batch (one batched matrix product per gate
+step), with optional Pauli-trajectory noise, readout error, shots and Parquet output with full provenance. It includes
+a `wq-batch` command and Slurm scripts for A100 clusters (`scripts/hpc/`).
+
+```bash
+pip install "westquant-cudaq[batched,gpu]"
+wq-batch run --manifest shard.jsonl --out shard.parquet --noise noise.json --trajectories 200 --shots 1000
+```
+
+See [docs/BATCHED.md](docs/BATCHED.md).
+
 ## Relationship to WestQuant Open
 
 - `westquant-core`: WQIR, RepGraph, plugin contracts, and generic search.

docs/BATCHED.md

@@ -0,0 +1,85 @@
+# Batched GPU simulation (`westquant_cudaq.batched`)
+
+Throughput-oriented state-vector simulation of **many small circuits** (about 2–24 qubits) for dataset generation.
+One batch holds thousands of circuits of the same width; each gate step is one batched matrix product for every
+row that shares the same target qubits, so the GPU stays busy even when each circuit is tiny.
+
+Status: v0.2.0. Validated against Qiskit `Statevector` and an exact density-matrix reference on CPU (NumPy).
+The GPU path runs the same array code through CuPy (cuBLAS batched GEMM); **run `bench_a100.sbatch` once on the
+cluster before a campaign** – it runs the test suite on the GPU and measures throughput per qubit count.
+
+## Install
+
+```bash
+pip install "westquant-cudaq[batched,gpu]"     # CUDA 12 + CuPy; CPU-only: [batched]
+```
+
+## Python
+
+```python
+from westquant_cudaq.batched import BatchRunner, PauliNoiseModel, from_qiskit, get_backend
+
+irs = [from_qiskit(qc, fusion="1q", circuit_id=f"c{i}", meta={"expected": "0101"}) for i, qc in enumerate(circuits)]
+noise = PauliNoiseModel.from_calibration({"gate_error_1q": 5e-4, "gate_error_2q": 8e-3, "readout_error": 0.015})
+runner = BatchRunner(backend=get_backend("cupy"), noise=noise, trajectories=200, shots=1000)
+runner.run_to_parquet(irs, "out.parquet")
+```
+
+`fusion`: `"full"` for ideal runs (fewest kernel calls), `"1q"` when noise is used (exact error placement).
+The CLI picks this automatically.
+
+## HPC workflow (Slurm, A100)
+
+Submit everything **from the repository root** (the scripts locate themselves via `$SLURM_SUBMIT_DIR`).
+
+1. **Edit `scripts/hpc/env.sh` once** (modules, venv path) and fill in `--account/--partition` in the `.sbatch` files.
+2. **Validate and benchmark on one GPU:** `sbatch scripts/hpc/bench_a100.sbatch $PWD`
+3. **Shard the circuits:**
+   `python scripts/hpc/make_manifest.py --circuits circuits.parquet --out $WORK/wq/shards --max-circuits 20000`
+   (input: JSONL/Parquet with `circuit_id`, `qasm`, optional `family`, `expected`, `meta`; or `--qasm-dir`).
+   Check the plan without a GPU: `wq-batch plan --manifest $WORK/wq/shards/shard_00000.jsonl --gpu-mem-gb 64`.
+4. **Run**, either as an array (one GPU per task)
+   `sbatch --array=0-15 scripts/hpc/run_array_a100.sbatch $WORK/wq/shards $WORK/wq/out --noise noise.json --trajectories 200 --shots 1000`
+   or on whole 4-GPU nodes
+   `sbatch --nodes=4 scripts/hpc/run_node_4xa100.sbatch $WORK/wq/shards $WORK/wq/out --noise noise.json --trajectories 200`.
+   Shard *i* goes to rank *i mod K*. Finished shards are skipped, so after a timeout you resubmit the same command.
+5. **Merge and check:**
+   `python scripts/hpc/merge_shards.py --shards $WORK/wq/shards --out-dir $WORK/wq/out --dataset $WORK/wq/dataset`
+   → `dataset/data/n_qubits=*/…parquet` and `dataset/campaign_summary.json`; exit code 1 if anything is missing.
+
+## Tuning on A100
+
+* **dtype:** `complex64` (default) halves memory and, since the kernels are memory-bound, typically gives close to 2× throughput versus `complex128`; the tests use
+  2e-5 tolerance. Use `complex128` for validation runs.
+* **Batch size** is set from free GPU memory: rows ≈ 0.6 × free / (2ⁿ × 8 B × 5). `--safety` changes the 0.6.
+  Noisy runs use `1 + trajectories` rows per circuit, so fewer circuits fit per batch.
+* **Where batching pays:** small n (≲ 14) is limited by per-step host work and kernel launches – batching gives the
+  largest gain there. Somewhere around n ≈ 20–24 a single circuit fills the GPU by itself; there, per-circuit simulation with CUDA-Q/cuStateVec is the better tool (measure the crossover with `bench_a100.sbatch`).
+* **Homogeneous batches are fastest.** Circuits with identical layouts (parameter sweeps, transpilation variants of
+  the same layout, trajectories) need one kernel call per gate and no gather/scatter. `make_manifest.py` sorts by
+  size; the runner sorts by structure within a batch.
+* **CPU cores** parse QASM in parallel (`--workers`, default `$SLURM_CPUS_PER_TASK`). If parsing dominates
+  (`parse_s` in the summary), raise `--cpus-per-task` or pre-convert circuits once.
+* If GPU utilisation is low for small n, run 2 processes per GPU (CUDA MPS) so one parses while the other simulates.
+
+## Noise model and limits
+
+Pauli trajectories: after each gate a random Pauli error with the gate's depolarizing probability; readout flips
+per clbit applied exactly to the averaged distribution. Average gate infidelity r is converted with
+p = r(d+1)/d. Not modelled (yet): idle decoherence (T1/T2), crosstalk, leakage, coherent errors, mid-circuit
+measurement and reset. Statistical error of the noisy distribution ~ 1/√trajectories.
+
+## Output columns (one row per circuit)
+
+`circuit_id, n_qubits, n_measured, n_ops, n_2q, structure_key, fusion, meta` · `ideal_/noisy_top_bitstrings`,
+`*_top_probs`, `*_probs` (full distribution when ≤ 10 measured bits), `*_success_prob` (if `expected` given),
+`hellinger_fidelity`, `counts` (if `--shots`) · provenance: `run_id, westquant_cudaq, qiskit, gpu_name, dtype,
+noise_digest, trajectories, seed, host, slurm_*`. Schema: `westquant_cudaq.batched.record_schema()`.
+
+## Roadmap
+
+* v0.3: cuStateVec batched backend (`custatevecApplyMatrixBatched`) behind the same API, compared against the CuPy
+  path on A100; device-side random numbers for trajectories; CUDA Graphs for repeated layouts.
+* v0.3: idle T1/T2 noise (Pauli-twirled thermal relaxation) from backend calibration.
+* Records map 1:1 onto the WQBench `hardware_runs`/`transitions` tables; link each campaign to an
+  `ExecutionPassport`.

pyproject.toml

@@ -4,13 +4,13 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "westquant-cudaq"
-version = "0.1.0"
-description = "Representation-aware CUDA-Q execution planning, GPU/QPU allocation, and reproducibility"
+version = "0.2.0"
+description = "Representation-aware CUDA-Q execution planning, GPU/QPU allocation, reproducibility, and batched GPU simulation for data generation"
 readme = "README.md"
 requires-python = ">=3.10"
 license = "Apache-2.0"
 authors = [{name = "WestQuant Open"}]
-keywords = ["quantum", "cuda-q", "cuquantum", "optimization", "scheduling", "gpu", "qpu"]
+keywords = ["quantum", "cuda-q", "cuquantum", "optimization", "scheduling", "gpu", "qpu", "simulation", "dataset"]
 classifiers = [
   "Development Status :: 3 - Alpha",
   "Intended Audience :: Science/Research",
@@ -25,17 +25,24 @@ dependencies = []
 [project.optional-dependencies]
 cudaq = ["cudaq>=0.12"]
 westquant = ["westquant-core>=0.2.0", "westquant-bridges>=0.1.0", "westquant-orchestrator>=0.1.0", "westquant-qcsc>=0.1.0"]
-test = ["pytest>=8"]
+# Batched simulator: CPU path (NumPy) + circuit import + Parquet output
+batched = ["numpy>=1.24", "qiskit>=1.0", "pyarrow>=14"]
+# GPU path for the batched simulator (CUDA 12; use cupy-cuda11x on CUDA 11, cupy-rocm-* on AMD)
+gpu = ["cupy-cuda12x>=13"]
+test = ["pytest>=8", "numpy>=1.24", "qiskit>=1.0", "pyarrow>=14"]
+
+[project.scripts]
+wq-batch = "westquant_cudaq.batched.cli:main"
 
 [project.urls]
 Homepage = "https://github.com/WestQuantOpen/westquant-cudaq"
 Documentation = "https://github.com/WestQuantOpen/westquant-cudaq#readme"
 Issues = "https://github.com/WestQuantOpen/westquant-cudaq/issues"
 
-[tool.setuptools]
-py-modules = ["westquant_cudaq"]
+[tool.setuptools.packages.find]
+include = ["westquant_cudaq*"]
 
 [tool.pytest.ini_options]
 addopts = "-q"
-testpaths = ["."]
+testpaths = ["tests"]
 python_files = ["test_*.py"]

scripts/hpc/bench_a100.sbatch

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Validate and benchmark on one A100 before a campaign: GPU test suite, then throughput per qubit count.
+#   sbatch scripts/hpc/bench_a100.sbatch /path/to/westquant-cudaq
+#SBATCH --job-name=wq-bench
+#SBATCH --account=YOUR_ACCOUNT
+#SBATCH --partition=YOUR_GPU_PARTITION
+#SBATCH --nodes=1
+#SBATCH --ntasks=1
+#SBATCH --gres=gpu:1
+#SBATCH --cpus-per-task=8
+#SBATCH --time=01:00:00
+#SBATCH --output=wq-bench_%j.out
+set -euo pipefail
+REPO=${1:-.}
+HERE=${WQ_HPC_DIR:-$SLURM_SUBMIT_DIR/scripts/hpc}
+source "$HERE/env.sh"
+nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv
+cd "$REPO"
+WQ_TEST_BACKEND=cupy python -m pytest -q tests/test_batched.py        # same tests as on CPU, now through CuPy
+for dt in complex64 complex128; do
+  echo "== heterogeneous random circuits, $dt"
+  wq-batch bench --backend cupy --dtype $dt --nmin 4 --nmax 20 --nstep 2 --circuits 4096 --depth 40 --compare-qiskit 32
+  echo "== same structure (parameter-sweep regime), $dt"
+  wq-batch bench --backend cupy --dtype $dt --nmin 4 --nmax 20 --nstep 4 --circuits 4096 --depth 40 --same-structure
+done

scripts/hpc/env.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+# Edit once per cluster. Sourced by the Slurm scripts.
+# Example for an NVIDIA A100 system with CUDA 12 modules (names differ per centre: check `module avail cuda python`).
+# module load python/3.11 cuda/12.2
+# One-time setup (login node):
+#   python -m venv $WORK/venvs/wq && source $WORK/venvs/wq/bin/activate
+#   pip install "westquant-cudaq[batched,gpu]"        # or: pip install -e /path/to/westquant-cudaq[batched,gpu]
+#   python -c "import cupy; print(cupy.cuda.runtime.getDeviceCount())"   # run inside a GPU job to verify
+source "${WQ_VENV:-$WORK/venvs/wq}/bin/activate"
+export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK:-8}
+export CUPY_CACHE_DIR=${CUPY_CACHE_DIR:-$WORK/.cupy_cache}   # kernel cache on shared FS, not $HOME quota

scripts/hpc/make_manifest.py

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""Split a circuit collection into GPU-sized, cost-balanced manifest shards.
+
+Input (one of):
+  --qasm-dir DIR        every *.qasm file is one circuit (circuit_id = file stem)
+  --circuits FILE       JSONL or Parquet with at least circuit_id and qasm (other columns are kept)
+
+Shards are sorted by qubit count, so a shard holds circuits of similar size and the simulator fills its batches well.
+Cost per circuit ~ 2**n * (number of gate lines); shards are cut when they reach --target-cost or --max-circuits.
+
+Writes OUT/shard_00000.jsonl ... and OUT/index.json (used by the Slurm scripts and merge_shards.py).
+
+  python make_manifest.py --circuits all_circuits.parquet --out shards --max-circuits 20000
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import re
+from pathlib import Path
+
+QREG = re.compile(r"qreg\s+\w+\s*\[\s*(\d+)\s*\]")
+QUBIT3 = re.compile(r"qubit\s*\[\s*(\d+)\s*\]")
+
+
+def n_qubits(qasm: str) -> int:
+    regs = [int(m) for m in QREG.findall(qasm)] or [int(m) for m in QUBIT3.findall(qasm)]
+    if regs:
+        return sum(regs)
+    from qiskit import qasm2
+    return qasm2.loads(qasm, custom_instructions=qasm2.LEGACY_CUSTOM_INSTRUCTIONS).num_qubits
+
+
+def gate_lines(qasm: str) -> int:
+    return sum(1 for line in qasm.splitlines()
+               if line.strip() and not line.lstrip().startswith(("OPENQASM", "include", "qreg", "creg", "qubit", "bit",
+                                                                  "measure", "barrier", "//")))
+
+
+def load(args) -> list[dict]:
+    if args.qasm_dir:
+        return [{"circuit_id": p.stem, "qasm": p.read_text()} for p in sorted(Path(args.qasm_dir).glob("*.qasm"))]
+    path = args.circuits
+    if path.endswith(".parquet"):
+        import pyarrow.parquet as pq
+        return pq.read_table(path).to_pylist()
+    with open(path) as f:
+        return [json.loads(line) for line in f if line.strip()]
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    src = ap.add_mutually_exclusive_group(required=True)
+    src.add_argument("--qasm-dir")
+    src.add_argument("--circuits")
+    ap.add_argument("--out", required=True)
+    ap.add_argument("--max-circuits", type=int, default=20000)
+    ap.add_argument("--target-cost", type=float, default=2e9, help="sum of 2**n * gates per shard")
+    ap.add_argument("--max-qubits", type=int, default=26, help="skip larger circuits (state-vector limit per row)")
+    args = ap.parse_args()
+
+    items = load(args)
+    seen, rows, skipped = set(), [], []
+    for it in items:
+        cid = str(it["circuit_id"])
+        if cid in seen:
+            raise SystemExit(f"duplicate circuit_id {cid}")
+        seen.add(cid)
+        n = n_qubits(it["qasm"])
+        if n > args.max_qubits:
+            skipped.append(cid)
+            continue
+        rows.append((n, (2 ** n) * max(1, gate_lines(it["qasm"])), it))
+    rows.sort(key=lambda r: (r[0], r[1]))
+
+    out = Path(args.out); out.mkdir(parents=True, exist_ok=True)
+    shards, cur, cost = [], [], 0.0
+    for n, c, it in rows:
+        if cur and (len(cur) >= args.max_circuits or cost + c > args.target_cost):
+            shards.append(cur); cur, cost = [], 0.0
+        cur.append((n, c, it)); cost += c
+    if cur:
+        shards.append(cur)
+
+    index = {"n_circuits": len(rows), "skipped_too_large": skipped, "shards": []}
+    for i, sh in enumerate(shards):
+        name = f"shard_{i:05d}.jsonl"
+        with open(out / name, "w") as f:
+            for _n, _c, it in sh:
+                f.write(json.dumps(it) + "\n")
+        index["shards"].append({"file": name, "circuits": len(sh), "n_min": sh[0][0], "n_max": sh[-1][0],
+                                "cost": sum(c for _n, c, _it in sh)})
+    (out / "index.json").write_text(json.dumps(index, indent=1))
+    print(f"{len(rows)} circuits -> {len(shards)} shards in {out} ({len(skipped)} skipped > {args.max_qubits} qubits)")
+
+
+if __name__ == "__main__":
+    main()