Evaluating Neural Video Compression and Super-Resolution for Low-Bitrate Autonomous Vehicle Teleoperation

Institut

Lehrstuhl für Fahrzeugtechnik (TUM-ED)

Typ

Masterarbeit /

Inhalt

 

Beschreibung

MOTIVATION

Autonomous vehicles are designed to drive independently, but they cannot handle every situation without support. In difficult or ambiguous cases, teleoperation can allow a remote operator to support the vehicle.

Teleoperation itself is challenging. The remote operator depends on video streams from the vehicle, which must provide sufficient visual information for decision-making while remaining low-latency and robust under limited bandwidth. This is especially relevant in low-bitrate situations, where video quality can degrade, and important scene information may become harder to interpret.

Classical video codecs have traditionally been used for this task. In this thesis, we want to evaluate whether neural video compression can offer advantages for low-bitrate autonomous vehicle teleoperation. An existing classical codec pipeline, a neural codec pipeline, and vehicle integration are already available. Building on this, the thesis will extend the neural compression approach with super-resolution and evaluate whether this improves teleoperation-relevant video quality under bandwidth constraints.

The work will include two studies: a replay-based study using critical teleoperation scenarios selected from an existing scenario pool, and conduct a live expert study with teleoperation on the EDGAR vehicle together.

 

Recommended Literature:

Neural Compression:

1. Nonlinear Transform Coding
2. End-to-End Neural Video Compression: A Review
3. DCVC-RT
4. Learned Low Bitrate Video Compression with Space-Time Super-Resolution

Teleoperation Studies:

1. Beyond VMAF: Towards Application-Specific Metrics for Teleoperation Video

2. Influence of Image Quality on the Teleoperation of Automated Vehicles

3. Enabling Teleoperated Driving in Everyday's Traffic Scenarios

 

Voraussetzungen

YOUR ROLE

The work packages of this thesis contain:

• Literature and Study Design: Review relevant work on teleoperation, HMI, video streaming, neural compression, and super-resolution. Based on this, design suitable evaluation studies for low-bitrate teleoperation.
• Scenario Selection: Select relevant critical teleoperation scenarios from an existing scenario pool for the initial offline study.
• Super-Resolution Extension: Implement and integrate a super-resolution extension for the existing neural compression pipeline.
• Offline Study: Set up and conduct a replay-based study comparing classical video compression, neural video compression, and neural compression with super-resolution in low-bitrate critical teleoperation scenarios.
• Live Expert Study: Set up and conduct a live expert study with teleoperation on the EDGAR vehicle under low-bitrate conditions.
• Evaluation: Analyze the results of both studies and compare the different compression approaches with respect to their suitability for low-bitrate autonomous vehicle teleoperation.

WHAT YOU SHOULD BRING ALONG

• Strong interest in autonomous driving and teleoperation
• Interest in HMI, machine learning, and video streaming
• Solid programming skills in Python
• Motivation to work with neural compression methods and real vehicle systems
• Independent and structured way of working

If you are interested in joining this project, feel free to send me an application with your CV and transcript of records. I look forward to receiving your application!

Email: niklas.krauss<script>document.write('@');</script>

<noscript>(at)</noscript>tum.de 

 

Verwendete Technologien

HMI,Neural Video Compression, Python, Pytorch, Machine Learning, Machine Learning, Super Resolution, Compression, Autonomous Driving, Autonomous Vehicles, Teleoperation

Tags

FTM Studienarbeit, FTM Krauss, FTM AV, FTM AV Safe Operation, FTM Informatik, FTM Teleoperation

Möglicher Beginn

sofort

Kontakt

Niklas Krauß
Raum: 3507
Tel.: +49172 1736882
niklas.krausstum.de