DynaSwitch Benchmark

Standardized Benchmarks for Non-Stationary Black-Box Optimization

DynaSwitch provides standardized benchmark environments and evaluation metrics for studying optimization algorithms under non-stationary conditions, abrupt regime shifts, and recurring environments.

---

1. Quick Start

git clone https://github.com/optimization-os/dyna-switch-benchmark.git
cd dyna-switch-benchmark
pip install -e .

# Run standard baseline verification (DE vs Random Search)
bash reproduce.sh

---

2. Benchmark Protocols

DynaSwitch evaluates two distinct non-stationary dynamics:

1. DynaSwitch (Single Abrupt Shift):
   • Evaluates fast recovery and plasticity when the objective function abruptly transforms (e.g. multimodal Rastrigin $\to$ ill-conditioned Rosenbrock valley).
2. RegimeReturn (Detour and Recurrence):
   • Evaluates memory retention vs catastrophic forgetting when an optimization process detours through an intermediate landscape and later returns to the initial regime (e.g. Rastrigin $\to$ Sphere $\to$ Rastrigin).

Baseline Reference Results (dim=20, budget=1500, 10 trials)

Environment	Random Search	Differential Evolution (DE)	OOPPG v3 (External Meta-Scheduler)	Key Metric Characteristic
DynaSwitch (Single Shift: Rastrigin $\to$ Rosenbrock)	125.53 ± 17.14	30.71 ± 14.58	42.98 ± 16.66	Static DE adapts faster than meta-schedulers on one-off shifts.
RegimeReturn (Recurrence: Rastrigin $\to$ Sphere $\to$ Rastrigin)	239.72 ± 14.11	280.44 ± 23.93	221.96 ± 18.41	DE suffers catastrophic forgetting; state preservation yields +20.9% error reduction over DE.

---

3. Environments

Environment	Class	Description
DynaSwitch	dyna_bench.environments.DynaSwitch	Step-gated abrupt landscape shift across registered objectives.
RegimeReturn	dyna_bench.environments.RegimeReturn	Three-phase detour benchmark evaluating catastrophic forgetting.
Stationary	dyna_bench.environments.stationary	Standard baseline landscapes (Sphere, Rastrigin, Rosenbrock, Ackley).

Metrics

Metric	Function	Description
Simple Regret	compute_regret	Distance from active optimum at step $t$.
Cumulative Regret	compute_cumulative_regret	Area under regret trajectory over budget.
Adaptation Speed	compute_adaptation_speed	Recovery time to reach pre-shift performance levels.

---

4. Usage Example

from dyna_bench.environments import DynaSwitch, RegimeReturn

# 1. Initialize environment with regime shift at step 500
env = DynaSwitch(dim=20, switch_points=[500], functions=['rastrigin', 'rosenbrock'])

# 2. Evaluate candidate points
for step in range(1000):
    x = sample_candidate()
    f_val = env.evaluate(x, step)
    regret = env.get_regret(x, step)

---

5. Citation

If you use DynaSwitch in your research:

@software{dynaswitch2026,
  author = {Yingjie Gao},
  title = {DynaSwitch: A Benchmark for Non-Stationary Black-Box Optimization with Regime Shifts},
  url = {https://github.com/optimization-os/dyna-switch-benchmark},
  year = {2026}
}

---

License

MIT License - see LICENSE for details.