Decentralized Coordination in Partially Observable Queueing Networks

Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review

Authors

  • Anam Tahir
  • Jiekai Jia
  • Heinz Koeppel

External Research Organisations

  • Technische Universität Darmstadt
View graph of relations

Details

Original languageEnglish
Title of host publicationGLOBECOM 2022-2022 IEEE Global Communications Conference
Pages1491-1496
Number of pages6
ISBN (electronic)978-1-6654-3540-6
Publication statusPublished - 4 Dec 2022
Externally publishedYes
Event2022 IEEE Global Communications Conference: Selected Areas in Communications: Machine Learning for Communications (GLOBECOM 2022) - Windsor Barra, Rio de Janeiro, Brazil
Duration: 4 Dec 2022 → 8 Dec 2022

Publication series

NameIEEE Global Communications Conference
ISSN (Print)1930-529X
ISSN (electronic)2576-6813

Abstract

We consider communication in a fully cooperative multi-agent system, where the agents have partial observation of the environment and must act jointly to maximize the overall reward. We have a discrete-time queueing network where agents route packets to queues based only on the partial information of the current queue lengths. The queues have limited buffer capacity, so packet drops happen when they are sent to a full queue. In this work, we implemented a communication channel for the agents to share their information in order to reduce the packet drop rate. For efficient information sharing we use an attention-based communication model, called ATVC, to select informative messages from other agents. The agents then infer the state of queues using a combination of the variational autoencoder, VAE, and product-of-experts, PoE, model. Ultimately, the agents learn what they need to communicate and with whom, instead of communicating all the time with everyone. We also show empirically that ATVC is able to infer the true state of the queues and leads to a policy which outperforms existing baselines.

Keywords

    communication, multi-agent system, queueing network, reinforcement learning

ASJC Scopus subject areas

Cite this

Decentralized Coordination in Partially Observable Queueing Networks. / Tahir, Anam; Jia, Jiekai; Koeppel, Heinz.
GLOBECOM 2022-2022 IEEE Global Communications Conference. 2022. p. 1491-1496 (IEEE Global Communications Conference).

Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review

Tahir, A, Jia, J & Koeppel, H 2022, Decentralized Coordination in Partially Observable Queueing Networks. in GLOBECOM 2022-2022 IEEE Global Communications Conference. IEEE Global Communications Conference, pp. 1491-1496, 2022 IEEE Global Communications Conference: Selected Areas in Communications, Rio de Janeiro, Brazil, 4 Dec 2022. https://doi.org/10.1109/GLOBECOM48099.2022.10001584, https://doi.org/10.48550/arXiv.2208.13621
Tahir, A., Jia, J., & Koeppel, H. (2022). Decentralized Coordination in Partially Observable Queueing Networks. In GLOBECOM 2022-2022 IEEE Global Communications Conference (pp. 1491-1496). (IEEE Global Communications Conference). https://doi.org/10.1109/GLOBECOM48099.2022.10001584, https://doi.org/10.48550/arXiv.2208.13621
Tahir A, Jia J, Koeppel H. Decentralized Coordination in Partially Observable Queueing Networks. In GLOBECOM 2022-2022 IEEE Global Communications Conference. 2022. p. 1491-1496. (IEEE Global Communications Conference). doi: 10.1109/GLOBECOM48099.2022.10001584, 10.48550/arXiv.2208.13621
Tahir, Anam ; Jia, Jiekai ; Koeppel, Heinz. / Decentralized Coordination in Partially Observable Queueing Networks. GLOBECOM 2022-2022 IEEE Global Communications Conference. 2022. pp. 1491-1496 (IEEE Global Communications Conference).
Download
@inproceedings{e22a0af9102b4e218537221225759b35,
title = "Decentralized Coordination in Partially Observable Queueing Networks",
abstract = "We consider communication in a fully cooperative multi-agent system, where the agents have partial observation of the environment and must act jointly to maximize the overall reward. We have a discrete-time queueing network where agents route packets to queues based only on the partial information of the current queue lengths. The queues have limited buffer capacity, so packet drops happen when they are sent to a full queue. In this work, we implemented a communication channel for the agents to share their information in order to reduce the packet drop rate. For efficient information sharing we use an attention-based communication model, called ATVC, to select informative messages from other agents. The agents then infer the state of queues using a combination of the variational autoencoder, VAE, and product-of-experts, PoE, model. Ultimately, the agents learn what they need to communicate and with whom, instead of communicating all the time with everyone. We also show empirically that ATVC is able to infer the true state of the queues and leads to a policy which outperforms existing baselines.",
keywords = "communication, multi-agent system, queueing network, reinforcement learning",
author = "Anam Tahir and Jiekai Jia and Heinz Koeppel",
note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE Global Communications Conference: Selected Areas in Communications : Machine Learning for Communications (GLOBECOM 2022), GLOBECOM 2022 ; Conference date: 04-12-2022 Through 08-12-2022",
year = "2022",
month = dec,
day = "4",
doi = "10.1109/GLOBECOM48099.2022.10001584",
language = "English",
isbn = "978-1-6654-3541-3",
series = "IEEE Global Communications Conference",
pages = "1491--1496",
booktitle = "GLOBECOM 2022-2022 IEEE Global Communications Conference",

}

Download

TY - GEN

T1 - Decentralized Coordination in Partially Observable Queueing Networks

AU - Tahir, Anam

AU - Jia, Jiekai

AU - Koeppel, Heinz

N1 - Publisher Copyright: © 2022 IEEE.

PY - 2022/12/4

Y1 - 2022/12/4

N2 - We consider communication in a fully cooperative multi-agent system, where the agents have partial observation of the environment and must act jointly to maximize the overall reward. We have a discrete-time queueing network where agents route packets to queues based only on the partial information of the current queue lengths. The queues have limited buffer capacity, so packet drops happen when they are sent to a full queue. In this work, we implemented a communication channel for the agents to share their information in order to reduce the packet drop rate. For efficient information sharing we use an attention-based communication model, called ATVC, to select informative messages from other agents. The agents then infer the state of queues using a combination of the variational autoencoder, VAE, and product-of-experts, PoE, model. Ultimately, the agents learn what they need to communicate and with whom, instead of communicating all the time with everyone. We also show empirically that ATVC is able to infer the true state of the queues and leads to a policy which outperforms existing baselines.

AB - We consider communication in a fully cooperative multi-agent system, where the agents have partial observation of the environment and must act jointly to maximize the overall reward. We have a discrete-time queueing network where agents route packets to queues based only on the partial information of the current queue lengths. The queues have limited buffer capacity, so packet drops happen when they are sent to a full queue. In this work, we implemented a communication channel for the agents to share their information in order to reduce the packet drop rate. For efficient information sharing we use an attention-based communication model, called ATVC, to select informative messages from other agents. The agents then infer the state of queues using a combination of the variational autoencoder, VAE, and product-of-experts, PoE, model. Ultimately, the agents learn what they need to communicate and with whom, instead of communicating all the time with everyone. We also show empirically that ATVC is able to infer the true state of the queues and leads to a policy which outperforms existing baselines.

KW - communication

KW - multi-agent system

KW - queueing network

KW - reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85146921710&partnerID=8YFLogxK

U2 - 10.1109/GLOBECOM48099.2022.10001584

DO - 10.1109/GLOBECOM48099.2022.10001584

M3 - Conference contribution

SN - 978-1-6654-3541-3

T3 - IEEE Global Communications Conference

SP - 1491

EP - 1496

BT - GLOBECOM 2022-2022 IEEE Global Communications Conference

T2 - 2022 IEEE Global Communications Conference: Selected Areas in Communications

Y2 - 4 December 2022 through 8 December 2022

ER -