Closing the Loop: Integrating Vision-Language Models for Feedback-Driven ADAS Scenario Generation

Institut

Professur für autonome Fahrzeugsysteme

Typ

Semesterarbeit / Masterarbeit /

Inhalt

 

Beschreibung

Background

Advanced Driver Assistance Systems (ADAS) rely heavily on large-scale scenario-based testing to ensure safety and robustness under a wide range of driving conditions. OpenSCENARIO has become a standardized format for describing such test cases in simulation environments. Recently, Large Language Models (LLMs) have shown promise in automatically generating diverse and complex ADAS test scenarios from high-level functional descriptions. However, a critical issue remains: there is often no clear mechanism to verify whether the generated scenarios accurately fulfill the original requirements. Manual validation is costly and does not scale with the increasing demand for scenario diversity.

Meanwhile, Vision-Language Models (VLMs), which combine visual and textual understanding, offer a powerful tool to semantically analyze simulated scenes. These models can interpret rendered scenarios, extract meaningful behavioral and spatial descriptions, and express them in natural language. This opens a new opportunity to introduce VLMs as intelligent validators in the scenario generation loop.

---

Objective

This work proposes a closed-loop scenario generation framework that integrates Vision-Language Models as scenario analyzers to validate and refine LLM-generated ADAS scenarios.  By combining LLMs for generation and VLMs for validation, this approach seeks to ensure that generated ADAS test scenarios are not only diverse and complex but also semantically correct and aligned with the intended use cases. This significantly enhances the safety assurance process for autonomous driving systems.

We Offer

• An exciting and forward-looking research environment.

• The opportunity to publish scientific results (subject to merit).

• Flexible supervision and the option to conduct the work in either German or English.

---

Requirements (What You Should Bring)

• Initiative and a creative, problem-solving mindset.

• Excellent proficiency in either German or English.

• Interest in autonomous driving, efficient deep learning models, or multimodal AI.

• Familiarity with computer vision or machine learning frameworks (e.g., PyTorch, TensorFlow) is advantageous.

• Familiarity with simulation frameworks (e.g., CARLA, SUMO, CommonRoad) is a plus.

• Basic understanding of finite state machines or formal methods is advantageous.

• Work can begin immediately. If you are interested in this topic, please have a look at our recent survey paper: https://arxiv.org/abs/2506.11526 and then send an email with a brief cover letter explaining why you are fascinated by this subject, along with a current transcript of records and your resume, to: yuan_avs.gao@tum.de.

 

Tags

AVS Gao

Möglicher Beginn

sofort

Kontakt

Yuan Gao
yuan_avs.gaotum.de