1. Theses & HiWi Jobs
    2.  

Vision-based End-to-End Autonomous Driving with Online Scene Reconstruction

Institute
Professur für autonome Fahrzeugsysteme
Type
Semester Thesis / Master's Thesis /
Content
 
Description

Modern autonomous vehicles increasingly rely on end-to-end learning systems that map raw sensor inputs directly to driving decisions. However, these systems often lack explicit understanding of scene semantics—such as road layout, centerlines, and obstacles—limiting their interpretability and generalization. To address this, we propose enhancing end-to-end autonomous driving with online scene reconstruction capabilities.

At the AVS Lab, we are building a vision-based driving policy that not only predicts future trajectories but also reconstructs scene representations (e.g., drivable area, centerlines, and road blockages) directly from camera inputs. These intermediate predictions are dynamically updated as new observations arrive, enabling situational awareness and more robust driving behavior.
Concrete examples can be found in HDMapNet and CenterLineDet.

Project Objective:
This project focuses on integrating online scene understanding into vision-based end-to-end driving. Core components include:

  • Training a perception backbone to extract scene structure (e.g., centerline heatmaps, occlusions, obstacles)
  • Fusing structured outputs with trajectory prediction for behavior planning
  • Exploring spatial-temporal consistency and online update mechanisms
  • Evaluating performance on interpretability, planning reliability, and driving safety

We offer:

  • Cutting-edge research combining perception and control
  • Close integration with an existing E2E driving stack
  • Opportunity to publish at top-tier robotics/AI venues
  • English-speaking environment, with thesis supervision possible

Your qualifications:

  • Solid Python and PyTorch skills
  • Interest in autonomous driving and deep learning
  • Strong background in deep learning and computer vision is a plus

Start date is flexible. If you're interested, please send:

  1. A highlight performance description of your computer science (e.g. deep learning, coding) related courses (can be within campus or online)
    (200 words would be enough)
  2. Academic transcript (optional for now, but may be required later)
  3. CV (optional)

to dingrui.wangtum.de.

Possible start
sofort
Contact
Dingrui Wang
dingrui.wangtum.de