Projects per year
Abstract
Bayesian optimization (BO) methods often rely on the assumption that the objective function is well-behaved, but in practice, this is seldom true for real-world objectives even if noise-free observations can be collected. Common approaches, which try to model the objective as precisely as possible, often fail to make progress by spending too many evaluations modeling irrelevant details. We address this issue by proposing surrogate models that focus on the well-behaved structure in the objective function, which is informative for search, while ignoring detrimental structure that is challenging to model from few observations. First, we demonstrate that surrogate models with appropriate noise distributions can absorb challenging structures in the objective function by treating them as irreducible uncertainty. Secondly, we show that a latent Gaussian process is an excellent surrogate for this purpose, comparing with Gaussian processes with standard noise distributions. We perform numerous experiments on a range of BO benchmarks and find that our approach improves reliability and performance when faced with challenging objective functions.
Original language | English |
---|---|
Title of host publication | 37th International Conference on Machine Learning, ICML 2020 |
Editors | Hal Daume, Aarti Singh |
Publisher | International Machine Learning Society (IMLS) |
Pages | 947-956 |
Number of pages | 10 |
ISBN (Electronic) | 9781713821120 |
Publication status | Published - 2020 |
Event | 37th International Conference on Machine Learning, ICML 2020 - Virtual, Online Duration: 13 Jul 2020 → 18 Jul 2020 |
Publication series
Name | 37th International Conference on Machine Learning, ICML 2020 |
---|---|
Volume | PartF168147-2 |
Conference
Conference | 37th International Conference on Machine Learning, ICML 2020 |
---|---|
City | Virtual, Online |
Period | 13/07/20 → 18/07/20 |
Bibliographical note
Funding Information:This project was supported by Engineering and Physical Sciences Research Council (EPSRC) Doctoral Training Partnership, the Hans Werthn Fund at The Royal Swedish Academy of Engineering Sciences, EPSRC CDE (EP/L016540/1), EPSRC CAMERA project (EP/M023281/1), the German Federal Ministry of Education and Research (project 01 IS 18049 A), and the Royal Society.
Publisher Copyright:
© 2020 37th International Conference on Machine Learning, ICML 2020. All rights reserved.
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
Funding
This project was supported by Engineering and Physical Sciences Research Council (EPSRC) Doctoral Training Partnership, the Hans Werthn Fund at The Royal Swedish Academy of Engineering Sciences, EPSRC CDE (EP/L016540/1), EPSRC CAMERA project (EP/M023281/1), the German Federal Ministry of Education and Research (project 01 IS 18049 A), and the Royal Society.
ASJC Scopus subject areas
- Computational Theory and Mathematics
- Human-Computer Interaction
- Software
Fingerprint
Dive into the research topics of 'Modulating surrogates for bayesian optimization'. Together they form a unique fingerprint.-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0
Campbell, N. (PI), Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Cosker, D. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Petrini, K. (CoI), Proulx, M. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/11/20 → 31/10/25
Project: Research council
-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Fincham Haines, T. (CoI), Hall, P. (CoI), Kim, K. I. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Richardt, C. (CoI), Salo, A. (CoI), Seminati, E. (CoI), Tabor, A. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council