Model-free reinforcement learning with noisy actions for automated experimental control in optics

Research output: Working paper/PreprintPreprint

View graph of relations

Details

Original languageUndefined/Unknown
Publication statusE-pub ahead of print - 24 May 2024

Abstract

Experimental control involves a lot of manual effort with non-trivial decisions for precise adjustments. Here, we study the automatic experimental alignment for coupling laser light into an optical fiber using reinforcement learning (RL). We face several real-world challenges, such as time-consuming training, partial observability, and noisy actions due to imprecision in the mirror steering motors. We show that we can overcome these challenges: To save time, we use a virtual testbed to tune our environment for dealing with partial observability and use relatively sample-efficient model-free RL algorithms like Soft Actor-Critic (SAC) or Truncated Quantile Critics (TQC). Furthermore, by fully training on the experiment, the agent learns directly to handle the noise present. In our extensive experimentation, we show that we are able to achieve 90% coupling, showcasing the effectiveness of our proposed approaches. We reach this efficiency, which is comparable to that of a human expert, without additional feedback loops despite the motors' inaccuracies. Our result is an example of the readiness of RL for real-world tasks. We consider RL a promising tool for reducing the workload in labs.

Keywords

    cs.LG, physics.optics, J.2; I.2.1

Cite this

Model-free reinforcement learning with noisy actions for automated experimental control in optics. / Richtmann, Lea; Schmiesing, Viktoria-S; Wilken, Dennis et al.
2024.

Research output: Working paper/PreprintPreprint

Richtmann, L, Schmiesing, V-S, Wilken, D, Heine, J, Tranter, A, Anand, A, Osborne, TJ & Heurs, M 2024 'Model-free reinforcement learning with noisy actions for automated experimental control in optics'.
Richtmann, L., Schmiesing, V.-S., Wilken, D., Heine, J., Tranter, A., Anand, A., Osborne, T. J., & Heurs, M. (2024). Model-free reinforcement learning with noisy actions for automated experimental control in optics. Advance online publication.
Richtmann L, Schmiesing VS, Wilken D, Heine J, Tranter A, Anand A et al. Model-free reinforcement learning with noisy actions for automated experimental control in optics. 2024 May 24. Epub 2024 May 24.
Richtmann, Lea ; Schmiesing, Viktoria-S ; Wilken, Dennis et al. / Model-free reinforcement learning with noisy actions for automated experimental control in optics. 2024.
Download
@techreport{207ee5e1a140440eb217992a266fdb77,
title = "Model-free reinforcement learning with noisy actions for automated experimental control in optics",
abstract = " Experimental control involves a lot of manual effort with non-trivial decisions for precise adjustments. Here, we study the automatic experimental alignment for coupling laser light into an optical fiber using reinforcement learning (RL). We face several real-world challenges, such as time-consuming training, partial observability, and noisy actions due to imprecision in the mirror steering motors. We show that we can overcome these challenges: To save time, we use a virtual testbed to tune our environment for dealing with partial observability and use relatively sample-efficient model-free RL algorithms like Soft Actor-Critic (SAC) or Truncated Quantile Critics (TQC). Furthermore, by fully training on the experiment, the agent learns directly to handle the noise present. In our extensive experimentation, we show that we are able to achieve 90% coupling, showcasing the effectiveness of our proposed approaches. We reach this efficiency, which is comparable to that of a human expert, without additional feedback loops despite the motors' inaccuracies. Our result is an example of the readiness of RL for real-world tasks. We consider RL a promising tool for reducing the workload in labs. ",
keywords = "cs.LG, physics.optics, J.2; I.2.1",
author = "Lea Richtmann and Viktoria-S Schmiesing and Dennis Wilken and Jan Heine and Aaron Tranter and Avishek Anand and Osborne, {Tobias J.} and Mich{\`e}le Heurs",
note = "10 pages + 10 pages appendices, 3 + 11 figures",
year = "2024",
month = may,
day = "24",
language = "Undefined/Unknown",
type = "WorkingPaper",

}

Download

TY - UNPB

T1 - Model-free reinforcement learning with noisy actions for automated experimental control in optics

AU - Richtmann, Lea

AU - Schmiesing, Viktoria-S

AU - Wilken, Dennis

AU - Heine, Jan

AU - Tranter, Aaron

AU - Anand, Avishek

AU - Osborne, Tobias J.

AU - Heurs, Michèle

N1 - 10 pages + 10 pages appendices, 3 + 11 figures

PY - 2024/5/24

Y1 - 2024/5/24

N2 - Experimental control involves a lot of manual effort with non-trivial decisions for precise adjustments. Here, we study the automatic experimental alignment for coupling laser light into an optical fiber using reinforcement learning (RL). We face several real-world challenges, such as time-consuming training, partial observability, and noisy actions due to imprecision in the mirror steering motors. We show that we can overcome these challenges: To save time, we use a virtual testbed to tune our environment for dealing with partial observability and use relatively sample-efficient model-free RL algorithms like Soft Actor-Critic (SAC) or Truncated Quantile Critics (TQC). Furthermore, by fully training on the experiment, the agent learns directly to handle the noise present. In our extensive experimentation, we show that we are able to achieve 90% coupling, showcasing the effectiveness of our proposed approaches. We reach this efficiency, which is comparable to that of a human expert, without additional feedback loops despite the motors' inaccuracies. Our result is an example of the readiness of RL for real-world tasks. We consider RL a promising tool for reducing the workload in labs.

AB - Experimental control involves a lot of manual effort with non-trivial decisions for precise adjustments. Here, we study the automatic experimental alignment for coupling laser light into an optical fiber using reinforcement learning (RL). We face several real-world challenges, such as time-consuming training, partial observability, and noisy actions due to imprecision in the mirror steering motors. We show that we can overcome these challenges: To save time, we use a virtual testbed to tune our environment for dealing with partial observability and use relatively sample-efficient model-free RL algorithms like Soft Actor-Critic (SAC) or Truncated Quantile Critics (TQC). Furthermore, by fully training on the experiment, the agent learns directly to handle the noise present. In our extensive experimentation, we show that we are able to achieve 90% coupling, showcasing the effectiveness of our proposed approaches. We reach this efficiency, which is comparable to that of a human expert, without additional feedback loops despite the motors' inaccuracies. Our result is an example of the readiness of RL for real-world tasks. We consider RL a promising tool for reducing the workload in labs.

KW - cs.LG

KW - physics.optics

KW - J.2; I.2.1

M3 - Preprint

BT - Model-free reinforcement learning with noisy actions for automated experimental control in optics

ER -

By the same author(s)