Defining the Cost Model for Auto-Tuning: Genetic Algorithms vs. Bayesian Optimization

[shreyas-omkar]

Defining the Cost Model for Auto-Tuning: Genetic Algorithms vs. Bayesian Optimization

Context

EdgeFlow is moving toward ML-based auto-tuning to optimize quantization parameters for edge devices.
A key design choice is the search strategy used to explore the parameter space efficiently.

Two leading approaches are:

• Genetic Algorithms (GA)
• Bayesian Optimization (BO)

Each has trade-offs in convergence speed, sample efficiency, and implementation complexity.

Discussion Points

• Which approach converges faster for small, noisy search spaces?
• How sensitive are GA and BO to inaccurate cost estimates?
• Which method is more suitable for heterogeneous hardware targets?
• Can either approach be made deterministic enough for CI testing?

Goal

Gather input from math and ML-focused contributors to guide the design of the Auto-Tuner GSoC project and avoid locking into a suboptimal optimization strategy early.