Learning to Forecast the Future: Physics-Informed Latent BEV World Models for Vision-based End to End Autonomous Driving
- Institut
- Professur für autonome Fahrzeugsysteme
- Typ
- Semesterarbeit Masterarbeit
- Inhalt
- Beschreibung
Autonomous driving continues to evolve by combining powerful perception, prediction, and planning systems. A critical part of this ecosystem is the vehicle’s ability to forecast its environment — not just in raw sensor space, but in a compact and structured format such as Bird’s-Eye View (BEV). Accurate future prediction in BEV is essential for robust planning and safe interaction with other agents on the road.
At the AVS Lab, we are exploring physics-informed latent world models that can predict future BEV scenes based on current observations. Inspired by recent advances in diffusion models and transformers, we aim to build a system that generates realistic, temporally consistent future BEV frames while embedding physical constraints such as kinematics and agent intent.
A concrete example can be find here.Project Objective:
This project investigates the use of generative models (e.g., Diffusion Transformer, Latent Auto-regressive Models) to predict future BEV representations over a short time horizon. Unlike purely data-driven models, we introduce physics-based priors (e.g., acceleration limits, yaw dynamics, map constraints) to improve realism and generalization. The focus lies on:- Building or extending a latent BEV world model
- Integrating physics constraints during training or inference
- Evaluating model quality in terms of realism, consistency, and physical feasibility
- Exploring downstream planning integration or multi-agent forecasting
We offer:
- Cutting-edge research in generative models and autonomous systems
- Integration with an existing end-to-end driving stack
- Opportunity to publish in top-tier conferences or journals
- Work in English, thesis supervision included
Your qualifications:
- Strong background in deep learning
- Familiarity with Python and PyTorch
- Understanding of vehicle dynamics or physics-based modeling
- Passion for autonomous driving and structured prediction tasks
Start date is flexible. If you're interested, please send:
- A highlight performance description of your computer science (e.g. deep learning, coding) related courses (can be within campus or online)
(200 words would be enough) - Academic transcript (optional for now, but may be required later)
- CV (optional)
- Möglicher Beginn
- sofort
- Kontakt
-
Dingrui Wang
dingrui.wangtum.de