TY - JOUR

T1 - Traffic Control Recognition with an Attention Mechanism Using Speed-Profile and Satellite Imagery data

AU - Cheng, Hao

AU - Lei, Haoran

AU - Zourlidou, Stefania

AU - Sester, Monika

N1 - Funding Information: This work is supported by the German Research Foundation (DFG) via the project GRK2159 i.c.sens.

PY - 2022/6/1

Y1 - 2022/6/1

N2 - Traffic regulators at intersections act as an essential factor that influences traffic flow and, subsequently, the route choices of commuters. A digital map that provides up-to-date traffic control information is beneficial not only for facilitating the commuters' trips, but also for energy-saving and environmental protection. In this paper, instead of using expensive surveying methods, we propose an automatic way based on a Conditional Variational Autoencoder (CVAE) to recognize traffic regulators, i. e., arm rules at intersections, by leveraging the GPS data collected from vehicles and the satellite imagery retrieved from digital maps, i. e., Google Maps. We apply a Long Short-Term Memory to extract the motion dynamics over a GPS sequence traversed through the intersection. Simultaneously, we build a Convolutional Neural Network (CNN) to extract the grid-based local imagery information associated with each step of the GPS positions. Moreover, a self-attention mechanism is adopted to extract the spatial and temporal features over both the GPS and grid sequences. The extracted temporal and spatial features are then combined for detecting the traffic arm rules. To analyze the performance of our method, we tested it on a GPS dataset collected by driving vehicles in Hannover, a medium-sized German city. Compared to a Random Forest model and an Encoder-Decoder model, our proposed model achieved better results with both accuracy and F1-score of 0.90 for the three-class (arm rules of uncontrolled, traffic light, and priority sign) task. We also carried out ablation studies to further investigate the effectiveness of the GPS input branch, the image input branch, and the self-attention mechanism in our model.

AB - Traffic regulators at intersections act as an essential factor that influences traffic flow and, subsequently, the route choices of commuters. A digital map that provides up-to-date traffic control information is beneficial not only for facilitating the commuters' trips, but also for energy-saving and environmental protection. In this paper, instead of using expensive surveying methods, we propose an automatic way based on a Conditional Variational Autoencoder (CVAE) to recognize traffic regulators, i. e., arm rules at intersections, by leveraging the GPS data collected from vehicles and the satellite imagery retrieved from digital maps, i. e., Google Maps. We apply a Long Short-Term Memory to extract the motion dynamics over a GPS sequence traversed through the intersection. Simultaneously, we build a Convolutional Neural Network (CNN) to extract the grid-based local imagery information associated with each step of the GPS positions. Moreover, a self-attention mechanism is adopted to extract the spatial and temporal features over both the GPS and grid sequences. The extracted temporal and spatial features are then combined for detecting the traffic arm rules. To analyze the performance of our method, we tested it on a GPS dataset collected by driving vehicles in Hannover, a medium-sized German city. Compared to a Random Forest model and an Encoder-Decoder model, our proposed model achieved better results with both accuracy and F1-score of 0.90 for the three-class (arm rules of uncontrolled, traffic light, and priority sign) task. We also carried out ablation studies to further investigate the effectiveness of the GPS input branch, the image input branch, and the self-attention mechanism in our model.

KW - Attention Mechanism

KW - Classification

KW - Deep Learning

KW - Generative Model

KW - Traffic Regulation

UR - http://www.scopus.com/inward/record.url?scp=85132154897&partnerID=8YFLogxK

U2 - 10.5194/isprs-archives-XLIII-B4-2022-287-2022

DO - 10.5194/isprs-archives-XLIII-B4-2022-287-2022

M3 - Conference article

AN - SCOPUS:85132154897

VL - 43

SP - 287

EP - 293

JO - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives

JF - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives

SN - 1682-1750

IS - B4-2022

T2 - 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission IV

Y2 - 6 June 2022 through 11 June 2022

ER -