3D Pedestrian tracking using local structure constraints

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Autoren

  • Uyen Nguyen
  • Christian Heipke
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Seiten (von - bis)347-358
Seitenumfang12
FachzeitschriftISPRS Journal of Photogrammetry and Remote Sensing
Jahrgang166
Frühes Online-Datum2 Juli 2020
PublikationsstatusVeröffentlicht - Aug. 2020

Abstract

Tracking pedestrians based on visual sensors has many diverse applications, among them autonomous driving. Besides obtaining high recall, maintaining the consistency of tracked trajectories during data association is one of the most crucial issues of any tracker. This issue has been tackled in the literature for some time, taking advantage of geometry cues for improving the pairwise matching of detections across consecutive frames. However, this idea has only been employed in a simple way and not thoroughly leveraged in existing studies, i.e., only 2D information is utilized that cannot help to completely understand the real-world geometry in 3D space. Motivated by this observation, in this paper, we present a new method called 3D-TLSR (3D pedestrian tracking using local structure refinement). We use stereo images and expand the idea of geometry cues into 3D space to improve the association of existing trajectories and new detections. We divide the assignment optimization into two steps: (1) determining trajectories whose assignments are strongly believed to be correct, which we call anchors and (2) employing geometry constraints between the anchors and their nearby trajectories in 3D space to improve the matching of less reliable assignments of the first step. In addition, we suggest a simple approach to compute and correct the velocity of a tracked person so that we can better recover missed detections. Experimental results on the well known KITTI tracking benchmark, the ETHMS data set, as well as a self-generated dataset show that our tracker yields comparable results to other state-of-the-art methods with (for KITTI) multi object tracking accuracy (MOTA) of 54.00, which is the best online result among all investigated approaches, multi object tracking precision (MOTP) of 73.03, which is the best of all reported values, and mostly tracked (MT) of 29.55, being the second-best result. On the ETHMS dataset, our approach obtains best results with large margins for recall, precision, and MT, while maintaining a reasonable low number of Id switches (IDs) and fragmentation (FG). These findings confirms the effectiveness of our proposed association method and velocity estimation approach.

ASJC Scopus Sachgebiete

Zitieren

3D Pedestrian tracking using local structure constraints. / Nguyen, Uyen; Heipke, Christian.
in: ISPRS Journal of Photogrammetry and Remote Sensing, Jahrgang 166, 08.2020, S. 347-358.

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Nguyen U, Heipke C. 3D Pedestrian tracking using local structure constraints. ISPRS Journal of Photogrammetry and Remote Sensing. 2020 Aug;166:347-358. Epub 2020 Jul 2. doi: 10.1016/j.isprsjprs.2020.05.002
Nguyen, Uyen ; Heipke, Christian. / 3D Pedestrian tracking using local structure constraints. in: ISPRS Journal of Photogrammetry and Remote Sensing. 2020 ; Jahrgang 166. S. 347-358.
Download
@article{e4cf2ef895e04015a627205697336527,
title = "3D Pedestrian tracking using local structure constraints",
abstract = "Tracking pedestrians based on visual sensors has many diverse applications, among them autonomous driving. Besides obtaining high recall, maintaining the consistency of tracked trajectories during data association is one of the most crucial issues of any tracker. This issue has been tackled in the literature for some time, taking advantage of geometry cues for improving the pairwise matching of detections across consecutive frames. However, this idea has only been employed in a simple way and not thoroughly leveraged in existing studies, i.e., only 2D information is utilized that cannot help to completely understand the real-world geometry in 3D space. Motivated by this observation, in this paper, we present a new method called 3D-TLSR (3D pedestrian tracking using local structure refinement). We use stereo images and expand the idea of geometry cues into 3D space to improve the association of existing trajectories and new detections. We divide the assignment optimization into two steps: (1) determining trajectories whose assignments are strongly believed to be correct, which we call anchors and (2) employing geometry constraints between the anchors and their nearby trajectories in 3D space to improve the matching of less reliable assignments of the first step. In addition, we suggest a simple approach to compute and correct the velocity of a tracked person so that we can better recover missed detections. Experimental results on the well known KITTI tracking benchmark, the ETHMS data set, as well as a self-generated dataset show that our tracker yields comparable results to other state-of-the-art methods with (for KITTI) multi object tracking accuracy (MOTA) of 54.00, which is the best online result among all investigated approaches, multi object tracking precision (MOTP) of 73.03, which is the best of all reported values, and mostly tracked (MT) of 29.55, being the second-best result. On the ETHMS dataset, our approach obtains best results with large margins for recall, precision, and MT, while maintaining a reasonable low number of Id switches (IDs) and fragmentation (FG). These findings confirms the effectiveness of our proposed association method and velocity estimation approach.",
keywords = "3D pedestrian tracking, Local structure constrains, Missed detection recovery, Online association",
author = "Uyen Nguyen and Christian Heipke",
note = "Funding Information: This work was supported by the German Research Foundation (DFG) as part of the Research Training Group i.c.sens [RTG2159]. ",
year = "2020",
month = aug,
doi = "10.1016/j.isprsjprs.2020.05.002",
language = "English",
volume = "166",
pages = "347--358",
journal = "ISPRS Journal of Photogrammetry and Remote Sensing",
issn = "0924-2716",
publisher = "Elsevier",

}

Download

TY - JOUR

T1 - 3D Pedestrian tracking using local structure constraints

AU - Nguyen, Uyen

AU - Heipke, Christian

N1 - Funding Information: This work was supported by the German Research Foundation (DFG) as part of the Research Training Group i.c.sens [RTG2159].

PY - 2020/8

Y1 - 2020/8

N2 - Tracking pedestrians based on visual sensors has many diverse applications, among them autonomous driving. Besides obtaining high recall, maintaining the consistency of tracked trajectories during data association is one of the most crucial issues of any tracker. This issue has been tackled in the literature for some time, taking advantage of geometry cues for improving the pairwise matching of detections across consecutive frames. However, this idea has only been employed in a simple way and not thoroughly leveraged in existing studies, i.e., only 2D information is utilized that cannot help to completely understand the real-world geometry in 3D space. Motivated by this observation, in this paper, we present a new method called 3D-TLSR (3D pedestrian tracking using local structure refinement). We use stereo images and expand the idea of geometry cues into 3D space to improve the association of existing trajectories and new detections. We divide the assignment optimization into two steps: (1) determining trajectories whose assignments are strongly believed to be correct, which we call anchors and (2) employing geometry constraints between the anchors and their nearby trajectories in 3D space to improve the matching of less reliable assignments of the first step. In addition, we suggest a simple approach to compute and correct the velocity of a tracked person so that we can better recover missed detections. Experimental results on the well known KITTI tracking benchmark, the ETHMS data set, as well as a self-generated dataset show that our tracker yields comparable results to other state-of-the-art methods with (for KITTI) multi object tracking accuracy (MOTA) of 54.00, which is the best online result among all investigated approaches, multi object tracking precision (MOTP) of 73.03, which is the best of all reported values, and mostly tracked (MT) of 29.55, being the second-best result. On the ETHMS dataset, our approach obtains best results with large margins for recall, precision, and MT, while maintaining a reasonable low number of Id switches (IDs) and fragmentation (FG). These findings confirms the effectiveness of our proposed association method and velocity estimation approach.

AB - Tracking pedestrians based on visual sensors has many diverse applications, among them autonomous driving. Besides obtaining high recall, maintaining the consistency of tracked trajectories during data association is one of the most crucial issues of any tracker. This issue has been tackled in the literature for some time, taking advantage of geometry cues for improving the pairwise matching of detections across consecutive frames. However, this idea has only been employed in a simple way and not thoroughly leveraged in existing studies, i.e., only 2D information is utilized that cannot help to completely understand the real-world geometry in 3D space. Motivated by this observation, in this paper, we present a new method called 3D-TLSR (3D pedestrian tracking using local structure refinement). We use stereo images and expand the idea of geometry cues into 3D space to improve the association of existing trajectories and new detections. We divide the assignment optimization into two steps: (1) determining trajectories whose assignments are strongly believed to be correct, which we call anchors and (2) employing geometry constraints between the anchors and their nearby trajectories in 3D space to improve the matching of less reliable assignments of the first step. In addition, we suggest a simple approach to compute and correct the velocity of a tracked person so that we can better recover missed detections. Experimental results on the well known KITTI tracking benchmark, the ETHMS data set, as well as a self-generated dataset show that our tracker yields comparable results to other state-of-the-art methods with (for KITTI) multi object tracking accuracy (MOTA) of 54.00, which is the best online result among all investigated approaches, multi object tracking precision (MOTP) of 73.03, which is the best of all reported values, and mostly tracked (MT) of 29.55, being the second-best result. On the ETHMS dataset, our approach obtains best results with large margins for recall, precision, and MT, while maintaining a reasonable low number of Id switches (IDs) and fragmentation (FG). These findings confirms the effectiveness of our proposed association method and velocity estimation approach.

KW - 3D pedestrian tracking

KW - Local structure constrains

KW - Missed detection recovery

KW - Online association

UR - http://www.scopus.com/inward/record.url?scp=85087281851&partnerID=8YFLogxK

U2 - 10.1016/j.isprsjprs.2020.05.002

DO - 10.1016/j.isprsjprs.2020.05.002

M3 - Article

AN - SCOPUS:85087281851

VL - 166

SP - 347

EP - 358

JO - ISPRS Journal of Photogrammetry and Remote Sensing

JF - ISPRS Journal of Photogrammetry and Remote Sensing

SN - 0924-2716

ER -