Tighter integration between analysis and julia_trotter subpackages

[BrendanKKrueger]

Notes written by Claude below. While I provided the basic ideas, Claude wrote the details on its own -- take the below description with a grain of salt.

⏺ Plan: Compare and Integrate julia_trotter with Python analysis Package

Background

Both packages compute overlapping quantities (matrices, eigenvalues, errors) but with different
implementations. The project already includes juliacall, enabling Python-Julia integration.

Part A: Comparison of Overlapping Features

Goal: Validate consistency and identify performance differences.

Test Cases

Use 3-4 Hamiltonians spanning 4-14 qubits to test scaling behavior.

Quantities to Compare

1. Hamiltonian Matrix Construction
   - Compare pauli_dict_to_matrix() (Python) vs build_sparse_hamiltonian() (Julia)
   - Metrics: Matrix norm difference, construction time
2. Eigenvalue Computation
   - Compare scipy.linalg.eigh() (Python) vs Julia's Arpack/Arnoldi eigensolvers
   - Metrics: Ground state energy agreement, computation time
   - Note: Julia computes partial spectrum; Python computes full spectrum
3. Trotter Unitary Construction
   - Build unitaries via Qualtran (Python) and reference_trotter_unitary() (Julia)
   - Metrics: Unitarity error ‖U†U - I‖, timing

Deliverable: Jupyter notebook with comparison results and performance summary.

Part B: Integration Plan

Goal: Enable Python to invoke Julia for improved performance and new capabilities.

Implementation

Create analysis/julia_bridge.py to wrap Julia functions:

• Load Julia modules via juliacall
• Expose key functions: compute_trotter_ground_energy(), compute_commutator_bounds()

Integration Points

1. Eigenvalue Computation (High Priority)
   - Add method='julia-arnoldi' option to _eigendecompose_full()
   - Use Julia for systems >10 qubits where speed matters
2. Commutator Error Bounds (New Feature)
   - Add config_analysis.enable_commutator_bounds option
   - Provides theoretical error predictions unavailable in Python
3. Configuration
   - Add use_julia_eigensolvers flag (default: False for backward compatibility)
   - Document when Julia provides advantages

Success Criteria

• Numerical agreement <1e-10 on overlapping quantities
• Julia >2x faster for eigenvalues on systems >12 qubits
• No breaking changes to existing workflows
• Tests pass for both Python-only and Julia-integrated modes