Network Channel based Reinforcement Learning for stable and robust Neural Video Streaming

Institut

Lehrstuhl für Fahrzeugtechnik (TUM-ED)

Typ

Masterarbeit /

Inhalt

 

Beschreibung

The aim of this work is to develop a simulated network environment and design a Reinforcement Learning (RL) agent to dynamically manage latent transmission in neural video compression, minimizing state drift under varying channel conditions.

MOTIVATION

In distributed neural video compression (e.g., streaming), both the encoder and decoder operate causally, relying on previously reconstructed frames as temporal context. Standard Rate-Distortion optimization assumes a perfect channel. However, in the real world, network capacity fluctuates and is subject to delays and packet loss. If a transmitted latent is corrupted or lost, the decoder’s reconstruction deviates from the encoder’s intended state. Because future frames are coded conditionally, this deviation propagates forward as severe "drift". Traditional standard codecs attempt to solve this using Forward Error Correction (FEC), which statically increases the overall bitrate even when the channel is not at full capacity.

To solve this, we propose a two-stage learning approach. A base neural video compression model, optimized for a perfect channel (Stage I), is already provided. The focus of this thesis is Stage II: formulating the problem as a Partially Observable Markov Decision Process (POMDP) and introducing a channel-aware RL agent. This agent will use a "World Model" to estimate channel capacity and will dynamically modulate transmission through a Rate Policy and a Control Policy to minimize the discrepancy between transmitted and received latents.

Voraussetzungen

YOUR ROLE

The work packages of this thesis contain:

• Environment Design: Extend an existing network emulation framework to work for reinforcement learning for the existing neural codec.
• RL Agent Development: Design and building a RL agent, that based on the observation space, environment model predicts the network state, implementation of a suitable policy problem for the control policy of the neural codec
• Reward Formulation: Implement and refine the reward function (r_t) to carefully balance latent drift penalties, network congestion (exceeding channel capacity), and successful throughput.
• Evaluation: Conduct a thorough evaluation in your simulated environment to determine if this problem is solvable via RL. Compare your agent's performance (in terms of stability, efficiency, and drift mitigation) against standard Forward Error Correction (FEC) baselines.

WHAT YOU SHOULD BRING ALONG

• Strong programming skills in Python and PyTorch
• Solid background in Reinforcement Learning and neural network architectures
• Understanding of computer networks and channel dynamics
• Initiative, strong mathematical formulation skills, and a research-oriented way of working
• Ideally: Basic knowledge of information theory and video compression principles

If you are interested in joining this project, feel free to send me an application with your CV and transcript of records. I look forward to receiving your application! 

EMail: niklas.krauss<script>document.write('@');</script>

<noscript>(at)</noscript>tum.de 

 

Verwendete Technologien

Neural Data Compression, Reinforcement Learning, Python, Pytorch, Machine Learning, Neural Video Compression, Autonomous Vehicles

Tags

FTM Studienarbeit, FTM Krauss, FTM AV, FTM AV Safe Operation, FTM Informatik, FTM Teleoperation

Möglicher Beginn

sofort

Kontakt

Niklas Krauß
Raum: 3507
Tel.: +49172 1736882
niklas.krausstum.de

Ausschreibung