# Optimization and Automated Machine Learning

Can we use Machine Learning techniques to improve Machine Learning processes themselves? Automated Machine Learning (AutoML) is about remoiving (some of) the human element from choosing ML parameters and methods. This gives rise to a difficult optimization problem where a single performance evaluation can take a long time, so fast convergence is desirable. Our group is therefore dealing with the following questions:

- How can we perform
**optimization as efficiently as possible**when single function evaluations are expensive? We tackle this “expensive black box optimization problem” with “Model Based Optimization”, sometimes called “Bayesian Optimization”, which itself relies on machine learning methods. - How can we use optimization methods to
**automatically improve Machine Learning methods**, given a problem at hand? For this, we may have to not only choose hyperparameters of a given Machine Learning algorithm, but instead choose this algorithm itself, in combination with possible preprocessing methods and/or methods of combining multiple algorithms (“Ensemble Methods”). - Does the
**AutoML method actually lead to better outcomes?**A problem that arises when optimization (by machines or, implicitly, by humans!) is performed is that the methods may perform well on the training data, but will generalize poorly when used in the wild. This not only arises when one algorithm is optimized on a training dataset and then performs worse on new unseen data, but even when we (the researches) develop a method that works well on our benchmark datasets but fails to work well when used for real-world applications. We therefore investigate ways to evaluate and compare AutoML methods on robust and meaningful benchmarks that tell us whether these methods are useful.

## Members

Name | Position | |||
---|---|---|---|---|

Dr. Janek Thomas | PostDoc | |||

Jakob Richter | PostDoc | |||

Martin Binder | PhD Student | |||

Stefan Coors | PhD Student | |||

Florian Karl | PhD Student | |||

Julia Moosbauer | PhD Student | |||

Florian Pfisterer | PhD Student | |||

Katharina Rath | PhD Student |

## Projects and Software

- mlrMBO: Model-based optimization with
`mlr`

. - AutoML Benchmark: Reproducible Benchmarks for AutoML Systems.
- mlr3hyperband: Multi-Armed Bandit Approach to Hyperparameter Tuning for
`mlr3`

. - AutoXGBoost: Automatic Tuning and Fitting of XGBoost.
- AutoXGBoost MC: Multi-Criterial Automatic Tuning and Fitting of XGBoost.
- mosmafs: Multi-Objective Simultaneous Model and Feature Selection.