Loading [MathJax]/extensions/tex2jax.js

Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Autorschaft

Details

OriginalspracheEnglisch
Titel des SammelwerksGenetic and Evolutionary Computation Conference (GECCO)
Seiten563 - 566
ISBN (elektronisch)9798400704956
PublikationsstatusVeröffentlicht - 1 Aug. 2024

Abstract

Dynamic Algorithm Configuration (DAC) addresses the challenge
of dynamically setting hyperparameters of an algorithm for a diverse set of instances rather than focusing solely on individual
tasks. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. However, the limited generalization performance of these agents has significantly hindered
the application in DAC. Our hypothesis is that a potential bias in
the training instances limits generalization capabilities. We take
a step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and then
retraining the agent on this subset to improve its generalization
performance. For constructing the meta-features for the subset selection, we particularly account for the dynamic nature of the RL
agent by computing time series features on trajectories of actions
and rewards generated by the agent’s interaction with the environment. Through empirical evaluations on the Sigmoid and CMA-ES
benchmarks from the standard benchmark library for DAC, called
DACBench, we discuss the potentials of our selection technique
compared to training on the entire instance set. Our results highlight the efficacy of instance selection in refining DAC policies for
diverse instance spaces.

Zitieren

Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization. / Benjamins, Carolin; Cenikj, Gjorgjina; Nikolikj, Ana et al.
Genetic and Evolutionary Computation Conference (GECCO). 2024. S. 563 - 566.

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Benjamins, C, Cenikj, G, Nikolikj, A, Mohan, A, Eftimov, T & Lindauer, M 2024, Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization. in Genetic and Evolutionary Computation Conference (GECCO). S. 563 - 566. https://doi.org/10.1145/3638530
Benjamins, C., Cenikj, G., Nikolikj, A., Mohan, A., Eftimov, T., & Lindauer, M. (2024). Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization. In Genetic and Evolutionary Computation Conference (GECCO) (S. 563 - 566) https://doi.org/10.1145/3638530
Benjamins C, Cenikj G, Nikolikj A, Mohan A, Eftimov T, Lindauer M. Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization. in Genetic and Evolutionary Computation Conference (GECCO). 2024. S. 563 - 566 doi: 10.1145/3638530
Benjamins, Carolin ; Cenikj, Gjorgjina ; Nikolikj, Ana et al. / Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization. Genetic and Evolutionary Computation Conference (GECCO). 2024. S. 563 - 566
Download
@inproceedings{06c8b71e14034825b349f573fed87aae,
title = "Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization",
abstract = "Dynamic Algorithm Configuration (DAC) addresses the challengeof dynamically setting hyperparameters of an algorithm for a diverse set of instances rather than focusing solely on individualtasks. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. However, the limited generalization performance of these agents has significantly hinderedthe application in DAC. Our hypothesis is that a potential bias inthe training instances limits generalization capabilities. We takea step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and thenretraining the agent on this subset to improve its generalizationperformance. For constructing the meta-features for the subset selection, we particularly account for the dynamic nature of the RLagent by computing time series features on trajectories of actionsand rewards generated by the agent{\textquoteright}s interaction with the environment. Through empirical evaluations on the Sigmoid and CMA-ESbenchmarks from the standard benchmark library for DAC, calledDACBench, we discuss the potentials of our selection techniquecompared to training on the entire instance set. Our results highlight the efficacy of instance selection in refining DAC policies fordiverse instance spaces.",
author = "Carolin Benjamins and Gjorgjina Cenikj and Ana Nikolikj and Aditya Mohan and Tome Eftimov and Marius Lindauer",
year = "2024",
month = aug,
day = "1",
doi = "10.1145/3638530",
language = "English",
pages = "563 -- 566",
booktitle = "Genetic and Evolutionary Computation Conference (GECCO)",

}

Download

TY - GEN

T1 - Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization

AU - Benjamins, Carolin

AU - Cenikj, Gjorgjina

AU - Nikolikj, Ana

AU - Mohan, Aditya

AU - Eftimov, Tome

AU - Lindauer, Marius

PY - 2024/8/1

Y1 - 2024/8/1

N2 - Dynamic Algorithm Configuration (DAC) addresses the challengeof dynamically setting hyperparameters of an algorithm for a diverse set of instances rather than focusing solely on individualtasks. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. However, the limited generalization performance of these agents has significantly hinderedthe application in DAC. Our hypothesis is that a potential bias inthe training instances limits generalization capabilities. We takea step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and thenretraining the agent on this subset to improve its generalizationperformance. For constructing the meta-features for the subset selection, we particularly account for the dynamic nature of the RLagent by computing time series features on trajectories of actionsand rewards generated by the agent’s interaction with the environment. Through empirical evaluations on the Sigmoid and CMA-ESbenchmarks from the standard benchmark library for DAC, calledDACBench, we discuss the potentials of our selection techniquecompared to training on the entire instance set. Our results highlight the efficacy of instance selection in refining DAC policies fordiverse instance spaces.

AB - Dynamic Algorithm Configuration (DAC) addresses the challengeof dynamically setting hyperparameters of an algorithm for a diverse set of instances rather than focusing solely on individualtasks. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. However, the limited generalization performance of these agents has significantly hinderedthe application in DAC. Our hypothesis is that a potential bias inthe training instances limits generalization capabilities. We takea step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and thenretraining the agent on this subset to improve its generalizationperformance. For constructing the meta-features for the subset selection, we particularly account for the dynamic nature of the RLagent by computing time series features on trajectories of actionsand rewards generated by the agent’s interaction with the environment. Through empirical evaluations on the Sigmoid and CMA-ESbenchmarks from the standard benchmark library for DAC, calledDACBench, we discuss the potentials of our selection techniquecompared to training on the entire instance set. Our results highlight the efficacy of instance selection in refining DAC policies fordiverse instance spaces.

U2 - 10.1145/3638530

DO - 10.1145/3638530

M3 - Conference contribution

SP - 563

EP - 566

BT - Genetic and Evolutionary Computation Conference (GECCO)

ER -

Von denselben Autoren