Details
| Original language | English |
|---|---|
| Article number | 9347828 |
| Pages (from-to) | 3055-3066 |
| Number of pages | 12 |
| Journal | IEEE Transactions on Pattern Analysis and Machine Intelligence |
| Volume | 43 |
| Issue number | 9 |
| Publication status | Published - 1 Sept 2021 |
| Externally published | Yes |
Abstract
Automated machine learning (AutoML) seeks to automatically find so-called machine learning pipelines that maximize the prediction performance when being used to train a model on a given dataset. One of the main and yet open challenges in AutoMLis an effective use of computational resources: An AutoML process involves the evaluation of many candidate pipelines, which are costly but often ineffective because they are canceled due to a timeout. In this paper, we present an approach to predict the runtime of two-step machine learning pipelines with up to one pre-processor, which can be used to anticipate whether or not a pipeline will time out. Separate runtime models are trained offline for each algorithm that may be used in a pipeline, and an overall prediction is derived from these models. We empirically show that the approach increases successful evaluations made by an AutoML tool while preserving or even improving on the previously best solutions.
Keywords
- Automated machine learning, hierarchical runtime prediction, runtime prediction for classifiers and pipelines
ASJC Scopus subject areas
- Computer Science(all)
- Software
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computational Theory and Mathematics
- Computer Science(all)
- Artificial Intelligence
- Mathematics(all)
- Applied Mathematics
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, No. 9, 9347828, 01.09.2021, p. 3055-3066.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Predicting Machine Learning Pipeline Runtimes in the Context of Automated Machine Learning
AU - Mohr, Felix
AU - Wever, Marcel
AU - Tornede, Alexander
AU - Hullermeier, Eyke
N1 - Publisher Copyright: © 1979-2012 IEEE.
PY - 2021/9/1
Y1 - 2021/9/1
N2 - Automated machine learning (AutoML) seeks to automatically find so-called machine learning pipelines that maximize the prediction performance when being used to train a model on a given dataset. One of the main and yet open challenges in AutoMLis an effective use of computational resources: An AutoML process involves the evaluation of many candidate pipelines, which are costly but often ineffective because they are canceled due to a timeout. In this paper, we present an approach to predict the runtime of two-step machine learning pipelines with up to one pre-processor, which can be used to anticipate whether or not a pipeline will time out. Separate runtime models are trained offline for each algorithm that may be used in a pipeline, and an overall prediction is derived from these models. We empirically show that the approach increases successful evaluations made by an AutoML tool while preserving or even improving on the previously best solutions.
AB - Automated machine learning (AutoML) seeks to automatically find so-called machine learning pipelines that maximize the prediction performance when being used to train a model on a given dataset. One of the main and yet open challenges in AutoMLis an effective use of computational resources: An AutoML process involves the evaluation of many candidate pipelines, which are costly but often ineffective because they are canceled due to a timeout. In this paper, we present an approach to predict the runtime of two-step machine learning pipelines with up to one pre-processor, which can be used to anticipate whether or not a pipeline will time out. Separate runtime models are trained offline for each algorithm that may be used in a pipeline, and an overall prediction is derived from these models. We empirically show that the approach increases successful evaluations made by an AutoML tool while preserving or even improving on the previously best solutions.
KW - Automated machine learning
KW - hierarchical runtime prediction
KW - runtime prediction for classifiers and pipelines
UR - http://www.scopus.com/inward/record.url?scp=85100855731&partnerID=8YFLogxK
U2 - 10.1109/tpami.2021.3056950
DO - 10.1109/tpami.2021.3056950
M3 - Article
C2 - 33539291
VL - 43
SP - 3055
EP - 3066
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
JF - IEEE Transactions on Pattern Analysis and Machine Intelligence
SN - 0162-8828
IS - 9
M1 - 9347828
ER -