Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Autorschaft

Externe Organisationen

  • Fachhochschule für die Wirtschaft (FHDW) Hannover
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Titel des SammelwerksAAMAS '24
UntertitelProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems
Herausgeber/-innenMehdi Dastani, Jaime Simao Sichman
Seiten1928-1937
Seitenumfang10
PublikationsstatusVeröffentlicht - 6 Mai 2024
Veranstaltung23rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2024 - Auckland, Neuseeland
Dauer: 6 Mai 202410 Mai 2024

Publikationsreihe

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
ISSN (Print)1548-8403

Abstract

Using synthesis- and AI-planning-based approaches, recent works investigated methods to support engineers with the automation of design, planning, and execution of multi-robot cells. However, real-time constraints and stochastic processes were not well covered due, e.g., to the high abstraction level of the problem modeling, and these methods do not scale well. In this paper, using probabilistic model checking, we construct a controller and integrate it with reinforcement learning approaches to synthesize the most efficient and correct multi-robot task schedules. Statistical Model Checking (SMC) is applied for system requirement verification. Our method is aware of uncertainties and considers robot movement times, interruption times, and stochastic interruptions that can be learned during multi-robot cell operations. We developed a model-at-runtime that integrates the execution of the production cell and optimizes its performance using a controller-based AI system. For this purpose and to derive the best policy, we implemented and compared AI-based methods, namely, Monte Carlo Tree Search, a heuristic AI-planning technique, and Q-learning, a model-free reinforcement learning method. Our results show that our methodology can choose time-efficient task sequences that consequently improve the cycle time and efficiently adapt to stochastic events, e.g., robot interruptions. Moreover, our approach scales well compared to previous investigations using SMC, which did not reveal any violation of the requirements.

ASJC Scopus Sachgebiete

Zitieren

Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning. / Wete, Eric; Greenyer, Joel; Kudenko, Daniel et al.
AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems. Hrsg. / Mehdi Dastani; Jaime Simao Sichman. 2024. S. 1928-1937 (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS).

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Wete, E, Greenyer, J, Kudenko, D & Nejdl, W 2024, Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning. in M Dastani & JS Sichman (Hrsg.), AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems. Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, S. 1928-1937, 23rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2024, Auckland, Neuseeland, 6 Mai 2024. https://doi.org/10.5555/3635637.3663056
Wete, E., Greenyer, J., Kudenko, D., & Nejdl, W. (2024). Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning. In M. Dastani, & J. S. Sichman (Hrsg.), AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (S. 1928-1937). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS). https://doi.org/10.5555/3635637.3663056
Wete E, Greenyer J, Kudenko D, Nejdl W. Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning. in Dastani M, Sichman JS, Hrsg., AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems. 2024. S. 1928-1937. (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS). doi: 10.5555/3635637.3663056
Wete, Eric ; Greenyer, Joel ; Kudenko, Daniel et al. / Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning. AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems. Hrsg. / Mehdi Dastani ; Jaime Simao Sichman. 2024. S. 1928-1937 (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS).
Download
@inproceedings{0f71931005ba432699e42fb2a1a8b552,
title = "Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning",
abstract = "Using synthesis- and AI-planning-based approaches, recent works investigated methods to support engineers with the automation of design, planning, and execution of multi-robot cells. However, real-time constraints and stochastic processes were not well covered due, e.g., to the high abstraction level of the problem modeling, and these methods do not scale well. In this paper, using probabilistic model checking, we construct a controller and integrate it with reinforcement learning approaches to synthesize the most efficient and correct multi-robot task schedules. Statistical Model Checking (SMC) is applied for system requirement verification. Our method is aware of uncertainties and considers robot movement times, interruption times, and stochastic interruptions that can be learned during multi-robot cell operations. We developed a model-at-runtime that integrates the execution of the production cell and optimizes its performance using a controller-based AI system. For this purpose and to derive the best policy, we implemented and compared AI-based methods, namely, Monte Carlo Tree Search, a heuristic AI-planning technique, and Q-learning, a model-free reinforcement learning method. Our results show that our methodology can choose time-efficient task sequences that consequently improve the cycle time and efficiently adapt to stochastic events, e.g., robot interruptions. Moreover, our approach scales well compared to previous investigations using SMC, which did not reveal any violation of the requirements.",
keywords = "Model Checking, Multi-robot Motion Planning, Multi-robot Task Planning, Q-Learning, Safe Reinforcement Learning",
author = "Eric Wete and Joel Greenyer and Daniel Kudenko and Wolfgang Nejdl",
note = "Publisher Copyright: {\textcopyright} 2024 International Foundation for Autonomous Agents and Multiagent Systems.; 23rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2024 ; Conference date: 06-05-2024 Through 10-05-2024",
year = "2024",
month = may,
day = "6",
doi = "10.5555/3635637.3663056",
language = "English",
isbn = "9798400704864",
series = "Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS",
pages = "1928--1937",
editor = "Mehdi Dastani and Sichman, {Jaime Simao}",
booktitle = "AAMAS '24",

}

Download

TY - GEN

T1 - Multi-Robot Motion and Task Planning in Automotive Production Using Controller-based Safe Reinforcement Learning

AU - Wete, Eric

AU - Greenyer, Joel

AU - Kudenko, Daniel

AU - Nejdl, Wolfgang

N1 - Publisher Copyright: © 2024 International Foundation for Autonomous Agents and Multiagent Systems.

PY - 2024/5/6

Y1 - 2024/5/6

N2 - Using synthesis- and AI-planning-based approaches, recent works investigated methods to support engineers with the automation of design, planning, and execution of multi-robot cells. However, real-time constraints and stochastic processes were not well covered due, e.g., to the high abstraction level of the problem modeling, and these methods do not scale well. In this paper, using probabilistic model checking, we construct a controller and integrate it with reinforcement learning approaches to synthesize the most efficient and correct multi-robot task schedules. Statistical Model Checking (SMC) is applied for system requirement verification. Our method is aware of uncertainties and considers robot movement times, interruption times, and stochastic interruptions that can be learned during multi-robot cell operations. We developed a model-at-runtime that integrates the execution of the production cell and optimizes its performance using a controller-based AI system. For this purpose and to derive the best policy, we implemented and compared AI-based methods, namely, Monte Carlo Tree Search, a heuristic AI-planning technique, and Q-learning, a model-free reinforcement learning method. Our results show that our methodology can choose time-efficient task sequences that consequently improve the cycle time and efficiently adapt to stochastic events, e.g., robot interruptions. Moreover, our approach scales well compared to previous investigations using SMC, which did not reveal any violation of the requirements.

AB - Using synthesis- and AI-planning-based approaches, recent works investigated methods to support engineers with the automation of design, planning, and execution of multi-robot cells. However, real-time constraints and stochastic processes were not well covered due, e.g., to the high abstraction level of the problem modeling, and these methods do not scale well. In this paper, using probabilistic model checking, we construct a controller and integrate it with reinforcement learning approaches to synthesize the most efficient and correct multi-robot task schedules. Statistical Model Checking (SMC) is applied for system requirement verification. Our method is aware of uncertainties and considers robot movement times, interruption times, and stochastic interruptions that can be learned during multi-robot cell operations. We developed a model-at-runtime that integrates the execution of the production cell and optimizes its performance using a controller-based AI system. For this purpose and to derive the best policy, we implemented and compared AI-based methods, namely, Monte Carlo Tree Search, a heuristic AI-planning technique, and Q-learning, a model-free reinforcement learning method. Our results show that our methodology can choose time-efficient task sequences that consequently improve the cycle time and efficiently adapt to stochastic events, e.g., robot interruptions. Moreover, our approach scales well compared to previous investigations using SMC, which did not reveal any violation of the requirements.

KW - Model Checking

KW - Multi-robot Motion Planning

KW - Multi-robot Task Planning

KW - Q-Learning

KW - Safe Reinforcement Learning

UR - http://www.scopus.com/inward/record.url?scp=85196424584&partnerID=8YFLogxK

U2 - 10.5555/3635637.3663056

DO - 10.5555/3635637.3663056

M3 - Conference contribution

AN - SCOPUS:85196424584

SN - 9798400704864

T3 - Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

SP - 1928

EP - 1937

BT - AAMAS '24

A2 - Dastani, Mehdi

A2 - Sichman, Jaime Simao

T2 - 23rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2024

Y2 - 6 May 2024 through 10 May 2024

ER -

Von denselben Autoren