Large Language Model for Intuitive Control of Robots in Micro-Assembly

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Authors

View graph of relations

Details

Original languageEnglish
Title of host publicationIEEE 20th International Conference on Automation Science and Engineering
Subtitle of host publicationCASE 2024
PublisherIEEE Computer Society
Pages3957-3962
Number of pages6
ISBN (electronic)9798350358513
ISBN (print)979-8-3503-5852-0
Publication statusPublished - 28 Aug 2024
Event20th IEEE International Conference on Automation Science and Engineering, CASE 2024 - Bari, Italy
Duration: 28 Aug 20241 Sept 2024

Publication series

NameIEEE International Conference on Automation Science and Engineering
ISSN (Print)2161-8070
ISSN (electronic)2161-8089

Abstract

In an era of rapid technological advances in microdevices and photonics, the importance of efficient automation solutions is becoming increasingly evident. In particular, the automation of assembly processes is gaining importance as a significant portion of costs is incurred in this phase. Programming robots, especially in micro-assembly, requires a high level of expertise due to the complex assembly systems and processes. With the rapid development of increasingly powerful Large Language Models (LLMs), their use for programming and controlling robots is becoming more and more prevalent. However, previous approaches have been limited to the field of service robots. In this paper, we present a framework that uses an LLM as an intuitive user interface for robot control in the field of micro-assembly. We integrate an LLM into a framework based on ROS2 (Robot Operation System 2), which enables skill-based control and programming of the micro-assembly robot. LLMs offer an intuitive access to the robot skills, thus facilitating robot control for users without knowledge of programming and robots. We demonstrate how an advanced LLM functions as an efficient interface, interpreting user instructions in context and initiating corresponding actions. Using a use case with exemplary user queries, we evaluate the performance of the implemented framework. Finally, we highlight further improvements and application possibilities.

ASJC Scopus subject areas

Cite this

Large Language Model for Intuitive Control of Robots in Micro-Assembly. / Wiemann, Rolf; Terei, Niklas; Raatz, Annika.
IEEE 20th International Conference on Automation Science and Engineering: CASE 2024. IEEE Computer Society, 2024. p. 3957-3962 (IEEE International Conference on Automation Science and Engineering).

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Wiemann, R, Terei, N & Raatz, A 2024, Large Language Model for Intuitive Control of Robots in Micro-Assembly. in IEEE 20th International Conference on Automation Science and Engineering: CASE 2024. IEEE International Conference on Automation Science and Engineering, IEEE Computer Society, pp. 3957-3962, 20th IEEE International Conference on Automation Science and Engineering, CASE 2024, Bari, Italy, 28 Aug 2024. https://doi.org/10.1109/CASE59546.2024.10711830
Wiemann, R., Terei, N., & Raatz, A. (2024). Large Language Model for Intuitive Control of Robots in Micro-Assembly. In IEEE 20th International Conference on Automation Science and Engineering: CASE 2024 (pp. 3957-3962). (IEEE International Conference on Automation Science and Engineering). IEEE Computer Society. https://doi.org/10.1109/CASE59546.2024.10711830
Wiemann R, Terei N, Raatz A. Large Language Model for Intuitive Control of Robots in Micro-Assembly. In IEEE 20th International Conference on Automation Science and Engineering: CASE 2024. IEEE Computer Society. 2024. p. 3957-3962. (IEEE International Conference on Automation Science and Engineering). doi: 10.1109/CASE59546.2024.10711830
Wiemann, Rolf ; Terei, Niklas ; Raatz, Annika. / Large Language Model for Intuitive Control of Robots in Micro-Assembly. IEEE 20th International Conference on Automation Science and Engineering: CASE 2024. IEEE Computer Society, 2024. pp. 3957-3962 (IEEE International Conference on Automation Science and Engineering).
Download
@inproceedings{36ec76481f92469a8f3b33c069e866c8,
title = "Large Language Model for Intuitive Control of Robots in Micro-Assembly",
abstract = "In an era of rapid technological advances in microdevices and photonics, the importance of efficient automation solutions is becoming increasingly evident. In particular, the automation of assembly processes is gaining importance as a significant portion of costs is incurred in this phase. Programming robots, especially in micro-assembly, requires a high level of expertise due to the complex assembly systems and processes. With the rapid development of increasingly powerful Large Language Models (LLMs), their use for programming and controlling robots is becoming more and more prevalent. However, previous approaches have been limited to the field of service robots. In this paper, we present a framework that uses an LLM as an intuitive user interface for robot control in the field of micro-assembly. We integrate an LLM into a framework based on ROS2 (Robot Operation System 2), which enables skill-based control and programming of the micro-assembly robot. LLMs offer an intuitive access to the robot skills, thus facilitating robot control for users without knowledge of programming and robots. We demonstrate how an advanced LLM functions as an efficient interface, interpreting user instructions in context and initiating corresponding actions. Using a use case with exemplary user queries, we evaluate the performance of the implemented framework. Finally, we highlight further improvements and application possibilities.",
author = "Rolf Wiemann and Niklas Terei and Annika Raatz",
note = "Publisher Copyright: {\textcopyright} 2024 IEEE.; 20th IEEE International Conference on Automation Science and Engineering, CASE 2024 ; Conference date: 28-08-2024 Through 01-09-2024",
year = "2024",
month = aug,
day = "28",
doi = "10.1109/CASE59546.2024.10711830",
language = "English",
isbn = "979-8-3503-5852-0",
series = "IEEE International Conference on Automation Science and Engineering",
publisher = "IEEE Computer Society",
pages = "3957--3962",
booktitle = "IEEE 20th International Conference on Automation Science and Engineering",
address = "United States",

}

Download

TY - GEN

T1 - Large Language Model for Intuitive Control of Robots in Micro-Assembly

AU - Wiemann, Rolf

AU - Terei, Niklas

AU - Raatz, Annika

N1 - Publisher Copyright: © 2024 IEEE.

PY - 2024/8/28

Y1 - 2024/8/28

N2 - In an era of rapid technological advances in microdevices and photonics, the importance of efficient automation solutions is becoming increasingly evident. In particular, the automation of assembly processes is gaining importance as a significant portion of costs is incurred in this phase. Programming robots, especially in micro-assembly, requires a high level of expertise due to the complex assembly systems and processes. With the rapid development of increasingly powerful Large Language Models (LLMs), their use for programming and controlling robots is becoming more and more prevalent. However, previous approaches have been limited to the field of service robots. In this paper, we present a framework that uses an LLM as an intuitive user interface for robot control in the field of micro-assembly. We integrate an LLM into a framework based on ROS2 (Robot Operation System 2), which enables skill-based control and programming of the micro-assembly robot. LLMs offer an intuitive access to the robot skills, thus facilitating robot control for users without knowledge of programming and robots. We demonstrate how an advanced LLM functions as an efficient interface, interpreting user instructions in context and initiating corresponding actions. Using a use case with exemplary user queries, we evaluate the performance of the implemented framework. Finally, we highlight further improvements and application possibilities.

AB - In an era of rapid technological advances in microdevices and photonics, the importance of efficient automation solutions is becoming increasingly evident. In particular, the automation of assembly processes is gaining importance as a significant portion of costs is incurred in this phase. Programming robots, especially in micro-assembly, requires a high level of expertise due to the complex assembly systems and processes. With the rapid development of increasingly powerful Large Language Models (LLMs), their use for programming and controlling robots is becoming more and more prevalent. However, previous approaches have been limited to the field of service robots. In this paper, we present a framework that uses an LLM as an intuitive user interface for robot control in the field of micro-assembly. We integrate an LLM into a framework based on ROS2 (Robot Operation System 2), which enables skill-based control and programming of the micro-assembly robot. LLMs offer an intuitive access to the robot skills, thus facilitating robot control for users without knowledge of programming and robots. We demonstrate how an advanced LLM functions as an efficient interface, interpreting user instructions in context and initiating corresponding actions. Using a use case with exemplary user queries, we evaluate the performance of the implemented framework. Finally, we highlight further improvements and application possibilities.

UR - http://www.scopus.com/inward/record.url?scp=85208226346&partnerID=8YFLogxK

U2 - 10.1109/CASE59546.2024.10711830

DO - 10.1109/CASE59546.2024.10711830

M3 - Conference contribution

AN - SCOPUS:85208226346

SN - 979-8-3503-5852-0

T3 - IEEE International Conference on Automation Science and Engineering

SP - 3957

EP - 3962

BT - IEEE 20th International Conference on Automation Science and Engineering

PB - IEEE Computer Society

T2 - 20th IEEE International Conference on Automation Science and Engineering, CASE 2024

Y2 - 28 August 2024 through 1 September 2024

ER -

By the same author(s)