Bayesian optimization

[maxfirmbach]

Hello QUEENS community,

I have recently started working with QUEENS and am already using it successfully for my optimization problems.

However, my forward-model evaluations are computationally expensive, so keeping the number of evaluations as low as possible is essential. I am therefore considering Bayesian optimization as an alternative to the currently available optimization methods (currently I use COBYLA).

Does anyone in the community have experience using Bayesian optimization together with QUEENS? Is there already an implementation or an established workflow for coupling QUEENS with an external (or internal?) Bayesian optimization library or code? I could not find corresponding functionality in the current documentation or codebase.

If Bayesian optimization is not yet supported, would there be general interest in adding such a feature?

Thank you very much for your help and insights!

Best regards,
Max

EDIT: Would this be a good starting point https://scikit-optimize.github.io/stable/modules/generated/skopt.gp_minimize.html#skopt.gp_minimize?

@sbrandstaeter @mayrmt

[sbrandstaeter]

Thanks for opening this discussion.

As far as I know, there is currently no dedicated Bayesian optimisation implementation in QUEENS — no iterator or module that does BO out of the box. What does exist, however, are essentially all the building blocks you would need: surrogate models, design-of-experiments/sampling, and the general iterator infrastructure. So for now you would have to assemble the BO loop yourself (acquisition function and the sequential update being the main missing pieces).

That said, I am convinced this is an extremely relevant topic for QUEENS. Expensive forward models are exactly the setting the framework is built for, so yes — there is general interest in having Bayesian optimisation supported, and in my opinion it is strategically important for the project.

It would be very welcome as a contribution, and I am happy to discuss and help in the design and implementation.

[maxfirmbach]

@sbrandstaeter Great! Let me think about this a bit more, maybe I can come up with an implementation 😉.