feat(qprog): unified measurement API via evaluate() + CircuitPreprocessor

[Shiro-Raven]

Summary

Unifies how quantum programs are measured behind a single public entry point. QuantumProgram.evaluate(params, preprocessor, ...) now drives every measurement routine, with named CircuitPreprocessors selecting cost / sample / metric behavior instead of callers assembling pipelines themselves. This replaces the old PipelineSet / transformations primitives.

What changed

• evaluate() + CircuitPreprocessor — one measurement entry point; cost_preprocessor() is now public and overridable (PCE overrides it with its counts-based variant).
• Result-key helpers consolidated into _result_keys_operations.py (merging transformations.py + _result_keys.py), with the reduce toolkit and extract_param_set_idx publicized.
• MeasurementStage — consolidated observable grouping and adaptive shot allocation; the analytic _backend_expval path is selected only when a backend supports expval and all circuits in the batch share the same observable(s).
• Trotter synthesis moved onto TrotterizationResult.synthesize_evolution.
• PCE now defaults to an entangling hardware-efficient ansatz instead of inheriting VQE's chemistry ansatz (which crashed with no electrons).
• suppress_performance_warnings knob threaded from QuantumProgram down to its pipelines.
• Docs + API reference refreshed to match: shot-allocation / dry-run / measurement guides, cross-page redundancy trimmed, StageInfo/DryRunReport fields documented, plus a batch of fixes surfaced by docs-only alpha/beta testing.

Testing

• Full suite green; docs build clean (0 warnings) with 190/190 RST snippets passing.
• pyrefly check: 0 errors.
• All 18 tutorials pass via tutorials/_ci_runner.py (local-maestro).

Notes for review

• optimizer is now required on VariationalQuantumAlgorithm (the implicit MonteCarloOptimizer() default was removed) — flagged in review; no deprecation window added (open to discussion if the implicit default shipped in a release).
• PipelineSet / divi.pipeline.transformations are removed in favor of evaluate() / CircuitPreprocessor / _result_keys_operations.

🤖 Generated with Claude Code

https://claude.ai/code/session_0186dAXah28i9CVMDsX9NvuC

[greptile-apps]

Greptile Summary

This PR unifies quantum program measurement behind a single QuantumProgram.evaluate(params, preprocessor) entry point and introduces CircuitPreprocessor as the selection mechanism for cost, sampling, and metric routines. The PipelineSet / transformations primitives are removed in favour of the new architecture, helper functions are consolidated into _result_keys_operations.py, ProgramState / OptimizerConfig are extracted into _program_state.py, and Trotter synthesis is moved to TrotterizationResult.synthesize_evolution.

• evaluate() + CircuitPreprocessor: One measurement entry point replaces _run_pipeline; preprocessors carry the post-spec transform, result format, optional terminal stage, and a cache_key for per-program pipeline memoisation.
• MeasurementStage rework: The _backend_expval auto-promotion no longer requires all circuits to carry a single observable; a new UserWarning is emitted when shot_distribution is set on an analytically-exact expval backend.
• PCE default ansatz: Defaults to a hardware-efficient GenericLayerAnsatz instead of inheriting VQE's chemistry ansatz (which crashed because PCE has no electrons).

Confidence Score: 5/5

The refactoring correctly maps all old pipeline paths to the new evaluate() + CircuitPreprocessor architecture; sign conventions in Trotter synthesis are faithfully preserved from the original _build_qiskit_circuit; PCE default-ansatz fix prevents a real crash.

All three inline comments are observational or suggest hardening — none describe a present defect in the changed logic. The new evaluate() collapse, the variance side-effect pattern, and the warning stacklevel are all functional but could be made more robust. Full test suite, 18 tutorials, and pyrefly clean run support merging.

divi/qprog/quantum_program.py (evaluate() key-collapse) and divi/qprog/variational_quantum_algorithm.py (_evaluate_cost_param_sets variance double-call) are worth a second look before the next release.

Important Files Changed

Filename	Overview
divi/qprog/quantum_program.py	New evaluate() entry point, _build_preprocessor_pipeline() with per-instance pipeline cache, _spec_stage() / _initial_spec() hooks replacing _build_pipelines(). total_circuit_count / total_run_time docstrings refreshed but implementations stay.
divi/qprog/variational_quantum_algorithm.py	Removes _build_pipelines / _expectation_pipeline; adds cost_preprocessor() (public, overridable), _preprocessors(), _post_spec_batch(). optimizer is now required; _argmin_finite guards against NaN/inf best-loss selection. meta_circuit_factories replaced by cost_circuit.
divi/pipeline/_preprocessor.py	New file. Defines CircuitPreprocessor dataclass and factory functions cost_preprocessor() / sample_preprocessor().
divi/pipeline/_result_keys_operations.py	New file consolidating result-key helpers from transformations.py and _result_keys.py; adds average_by_param_set, group_by_branch_and_param_set, typed error variants in reduce_mean/reduce_merge_histograms.
divi/pipeline/stages/_measurement_stage.py	Relaxes _backend_expval single-observable restriction; adds UserWarning when shot_distribution is set on an analytically-exact backend; imports moved from deleted transformations.py.
divi/hamiltonians/_trotterization.py	Adds TrotterizationResult.synthesize_evolution, moving circuit-building logic out of TimeEvolution._build_qiskit_circuit; sign conventions mirror the original code exactly.
divi/qprog/algorithms/_pce.py	Replaces _build_pipelines override with cached_property _pce_cost_preprocessor + cost_preprocessor() override; defaults to GenericLayerAnsatz to avoid crash on no-electron programs; _create_meta_circuit_factories → _create_cost_circuit.
divi/qprog/_metrics.py	Refactors metric evaluation to use evaluate() + CircuitPreprocessor; pullback metric now averages branch-level metrics (E_b[G_b]) for multi-branch cost cohorts (QDrift); Fubini-Study per-block logic ported to branch-aware path.
divi/qprog/_solution_sampling_mixin.py	Removes _build_pipelines cooperative override and init_subclass MRO guard; replaces with _preprocessors() / _sample_preprocessor(); sample_solution resolves None params via _resolve_sample_params hook.
divi/pipeline/stages/_preprocess_stage.py	New stage that applies a CircuitPreprocessor's transform to every MetaCircuit in the post-spec batch.
divi/qprog/_program_state.py	New private module housing ProgramState, OptimizerConfig, and SubclassState extracted from variational_quantum_algorithm.py; re-exported at the old module level.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Optimizer
    participant VQA as VariationalQuantumAlgorithm
    participant QP as QuantumProgram
    participant Pipeline as CircuitPipeline
    participant Spec as SpecStage
    participant Preprocess as PreprocessStage
    participant Measure as MeasurementStage

    Optimizer->>VQA: cost_fn(params)
    VQA->>VQA: _evaluate_cost_param_sets(param_sets)
    VQA->>QP: evaluate(params, cost_preprocessor())
    QP->>QP: _build_preprocessor_pipeline(preprocessor)
    QP->>Pipeline: assemble [spec → preprocess → qem → measure]
    QP->>Pipeline: "run(initial_spec=cost_circuit, env)"
    Pipeline->>Spec: expand(cost_circuit)
    Spec-->>Pipeline: "{ham_key: MetaCircuit}"
    Pipeline->>Preprocess: expand(batch)
    Note over Preprocess: identity transform for cost
    Preprocess-->>Pipeline: "{ham_key: MetaCircuit}"
    Pipeline->>Measure: expand(batch)
    Measure-->>Pipeline: "{param_set_key: expval}"
    Pipeline-->>QP: PipelineResult
    QP->>QP: "collapse to {param_set_idx: value}"
    QP-->>VQA: dict[int, float]
    VQA-->>Optimizer: losses

    Note over Optimizer,VQA: For sample_solution()
    Optimizer->>VQA: sample_solution(params)
    VQA->>QP: evaluate(params, sample_preprocessor())
    QP->>Pipeline: "run(initial_spec=cost_circuit, env)"
    Pipeline->>Spec: expand(cost_circuit)
    Spec-->>Pipeline: "{ham_key: MetaCircuit w/ observable}"
    Pipeline->>Preprocess: expand(batch)
    Note over Preprocess: _clear_observable → measured_wires set
    Preprocess-->>Pipeline: "{ham_key: MetaCircuit w/o observable}"
    Pipeline->>Measure: expand(batch)
    Measure-->>Pipeline: "{param_set_key: probs}"
    Pipeline-->>QP: PipelineResult
    QP-->>VQA: dict[int, dict[str,float]]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Optimizer
    participant VQA as VariationalQuantumAlgorithm
    participant QP as QuantumProgram
    participant Pipeline as CircuitPipeline
    participant Spec as SpecStage
    participant Preprocess as PreprocessStage
    participant Measure as MeasurementStage

    Optimizer->>VQA: cost_fn(params)
    VQA->>VQA: _evaluate_cost_param_sets(param_sets)
    VQA->>QP: evaluate(params, cost_preprocessor())
    QP->>QP: _build_preprocessor_pipeline(preprocessor)
    QP->>Pipeline: assemble [spec → preprocess → qem → measure]
    QP->>Pipeline: "run(initial_spec=cost_circuit, env)"
    Pipeline->>Spec: expand(cost_circuit)
    Spec-->>Pipeline: "{ham_key: MetaCircuit}"
    Pipeline->>Preprocess: expand(batch)
    Note over Preprocess: identity transform for cost
    Preprocess-->>Pipeline: "{ham_key: MetaCircuit}"
    Pipeline->>Measure: expand(batch)
    Measure-->>Pipeline: "{param_set_key: expval}"
    Pipeline-->>QP: PipelineResult
    QP->>QP: "collapse to {param_set_idx: value}"
    QP-->>VQA: dict[int, float]
    VQA-->>Optimizer: losses

    Note over Optimizer,VQA: For sample_solution()
    Optimizer->>VQA: sample_solution(params)
    VQA->>QP: evaluate(params, sample_preprocessor())
    QP->>Pipeline: "run(initial_spec=cost_circuit, env)"
    Pipeline->>Spec: expand(cost_circuit)
    Spec-->>Pipeline: "{ham_key: MetaCircuit w/ observable}"
    Pipeline->>Preprocess: expand(batch)
    Note over Preprocess: _clear_observable → measured_wires set
    Preprocess-->>Pipeline: "{ham_key: MetaCircuit w/o observable}"
    Pipeline->>Measure: expand(batch)
    Measure-->>Pipeline: "{param_set_key: probs}"
    Pipeline-->>QP: PipelineResult
    QP-->>VQA: dict[int, dict[str,float]]

Loading

_{Reviews (2): Last reviewed commit: "feat(qprog): unified measurement API via..." | Re-trigger Greptile}

[Shiro-Raven]

@greptile-apps — responding to your review: two of these findings aren't being changed. Recording the rationale here since there are no inline threads for them.

Greptile flag — _build_pipelines removed without a deprecation note — intentional. _build_pipelines was a private (underscore) extension point; it's replaced by the cooperative _preprocessors() / _initial_spec() / _spec_stage() hooks. We don't run deprecation cycles for private internals, so no shim or migration note is being added. (run/has_results stay abstract; the pipeline hooks are deliberately non-abstract so a program that never calls evaluate() can opt out and still get dry_run() == {}.)

Greptile flag — SolutionSamplingMixin.__init_subclass__ MRO guard removed — obsolete, removed deliberately. That guard protected the old _build_pipelines chain, where VariationalQuantumAlgorithm._build_pipelines was a non-cooperative terminal, so a misordered MRO bypassed the mixin and dropped the sample pipeline. The new _preprocessors() chain is fully cooperative — every level does (*super()._preprocessors(), …) down to QuantumProgram._preprocessors() -> () — so a misordered MRO only reorders the returned tuple, never drops a routine, and dry_run() keys reports by name regardless of order. The guard now protects a failure mode that can no longer occur.