Attribute-Guided Reasoning for Motion Prediction in Autonomous Driving

Institut

Lehrstuhl für Fahrzeugtechnik (TUM-ED)

Typ

Semesterarbeit / Masterarbeit /

Inhalt

experimentell / theoretisch /  

Beschreibung

Motion prediction of surrounding agents is a key requirement for autonomous driving, espe cially for agents whose future behavior is hard to anticipate. In previous work, an end-to-end driving model (based on NVIDIA's Alpamayo) was repurposed as a motion predictor for a sin gle agent of interest that is not the ego vehicle. The model takes images, map data, and the agent's history as input. A reasoning backbone estimates attributes describing the agent's be haviour, and a decoder uses these attributes to condition the predicted trajectory. 

Evaluation of this baseline shows that the trajectory quality is limited mainly by the reasoning backbone: the estimated attributes are often inaccurate or poorly grounded in the scene, which in turn limits the trajectories the decoder can produce. Reasoning quality is therefore the main bottleneck of the current approach. This thesis focuses on improving the reasoning backbone of this approach. 

The goal is first to understand why the current attribute estimates are often inaccurate or poorly grounded, and then to develop and evaluate ways to make the reasoning more reliable, for example through improved training data, supervision, or reasoning strategies. The resulting reasoning backbone should produce attribute estimates that are better grounded in the scene, and thereby improve the final trajectory prediction, especially for the difficult, long-tail agents that motivated this work, compared to the current baseline. 

+ Publication in case of excellent work.

Work packages:

• Literature review on reasoning-based motion prediction and ways to improve reasoning in vision-language(-action) models. 
• Analysis of the current reasoning backbone's attribute estimation errors. 
• Development of an improved reasoning backbone. 
• Integration of the improved reasoning backbone with the existing decoder, including fine-tuning. 
• In-depth evaluation against the baseline, covering attribute and trajectory quality.

Voraussetzungen

Requirements:

• Very good programming skills in Python
• Experience with or strong interest in vision-language(-action) models or large language models. 
• High personal motivation and independent working style. 
• Very good language proficiency in German, English or French.

Software

Python, C++, ROS2

Tags

FTM AV, FTM AV Perception, FTM Stratil, FTM Informatik

Möglicher Beginn

sofort

Kontakt

Loïc Stratil, M.Sc.
Raum: MW 3508
Tel.: +49.89.289.15898
loic.stratiltum.de

Ausschreibung