TY - GEN

T1 - RABBIT

T2 - Live Transcoding of V-PCC Point Cloud Streams

AU - Rudolph, Michael

AU - Schneegass, Stefan

AU - Rizk, Amr

N1 - Publisher Copyright: © 2023 ACM.

PY - 2023/6/7

Y1 - 2023/6/7

N2 - Point clouds are a mature representation format for volumetric objects in 6 degrees-of-freedom multimedia streaming. To handle the massive size of point cloud data for visually satisfying immersive media, MPEG standardized Video-based Point Cloud Compression (V-PCC), leveraging existing video codecs to achieve high compression ratios. A major challenge of V-PCC is the high encoding latency, which results in fallback solutions that exchange the compression ratio for faster point cloud codecs. This encoding effort rises significantly in adaptive streaming systems, where heterogeneous user requirements translate into a set of quality representations of the media.In this paper, we show that given one high quality media representation we can achieve live transcoding of video-based compressed point clouds to serve heterogeneous user quality requirements in real time. This stands in contrast to the slow, baseline transcoding that reconstructs and re-encodes the raw point cloud at a new quality setting. To address the high latency when employing the decoder-encoder stack of V-PCC during transcoding, we propose RABBIT, a novel technique that only re-encodes the underlying video sub-streams. This eliminates the overhead of the baseline decoding-encoding approach and decreases the latency further by applying optimized video codecs. We perform extensive evaluation of RABBIT in combination with different video codecs, showing on-par quality with the baseline V-PCC transcoding. Using a hardware-Accelerated video codec we demonstrate live transcoding performance of RABBIT and finally present a trade-off between rate, distortion and transcoding latency.

AB - Point clouds are a mature representation format for volumetric objects in 6 degrees-of-freedom multimedia streaming. To handle the massive size of point cloud data for visually satisfying immersive media, MPEG standardized Video-based Point Cloud Compression (V-PCC), leveraging existing video codecs to achieve high compression ratios. A major challenge of V-PCC is the high encoding latency, which results in fallback solutions that exchange the compression ratio for faster point cloud codecs. This encoding effort rises significantly in adaptive streaming systems, where heterogeneous user requirements translate into a set of quality representations of the media.In this paper, we show that given one high quality media representation we can achieve live transcoding of video-based compressed point clouds to serve heterogeneous user quality requirements in real time. This stands in contrast to the slow, baseline transcoding that reconstructs and re-encodes the raw point cloud at a new quality setting. To address the high latency when employing the decoder-encoder stack of V-PCC during transcoding, we propose RABBIT, a novel technique that only re-encodes the underlying video sub-streams. This eliminates the overhead of the baseline decoding-encoding approach and decreases the latency further by applying optimized video codecs. We perform extensive evaluation of RABBIT in combination with different video codecs, showing on-par quality with the baseline V-PCC transcoding. Using a hardware-Accelerated video codec we demonstrate live transcoding performance of RABBIT and finally present a trade-off between rate, distortion and transcoding latency.

KW - 6DOF

KW - adaptive streaming

KW - point cloud

KW - virtual reality

UR - http://www.scopus.com/inward/record.url?scp=85163679403&partnerID=8YFLogxK

U2 - 10.1145/3587819.3590978

DO - 10.1145/3587819.3590978

M3 - Conference contribution

SP - 97

EP - 107

BT - MMSys '23

PB - Association for Computing Machinery (ACM)

ER -