[SA/IDP/MA] LLM/VLM-Assisted Adversarial Scenario Generation and Editing for Autonomous Driving Simulation
- Institut
- Lehrstuhl für Fahrzeugtechnik (TUM-ED)
- Typ
- Semesterarbeit Masterarbeit
- Inhalt
- experimentell theoretisch
- Beschreibung
Motivation
Constructing safety-critical scenarios manually is highly labor-intensive and lacks scalability. To automate this process, this project proposes an intelligent scenario generation and editing framework leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs).
Unlike purely text-based instructions, we aim to develop a framework that ingests structured environment representations, including HD maps and surrounding dynamic object information. The core objective is to programmatically transform normal driving scenarios into challenging adversarial situations.
- Voraussetzungen
Work Packages
In this project, you will build a unified pipeline that automatically interprets normal driving layouts and crafts complex, adversarial edge cases. The work packages are organized as follows:
Literature Review: Investigating state-of-the-art LLM and VLM-based critical/adversarial scenario generation techniques.
Adversarial Editing Engine & Action Generator: Developing the core reasoning engine that changes other vehicle trajectories and outputs safety-critical maneuvers.
Adversarial Scenario Dataset: generating structured datasets of adversarial scenarios in formats like CommonRoad or OpenSCENARIO, establishing a standardized database of corner cases for automated benchmarking.
Verification through CARLA/esmini Execution: Executing and validating the generated adversarial scenarios in dual simulation environments.
What you should bring along?
Very good programming skills in Python and PyTorch.
Familiarity with foundation models (LLMs/VLMs) and tools (CARLA, esmini, CommonRoad, OpenScenario) is a plus.
High personal motivation and independent working style.
Very good language proficiency in English.
Recommended literature:
Possibility for publication in case of excellent work.
If you are interested, please send me a grade sheet, your CV, short introduction (~5 sentences why this topic is interesting to you), and earliest possible date!
- Tags
- FTM Studienarbeit, FTM AV, FTM AV Perception, FTM Lim, FTM Informatik, FTM IDP
- Möglicher Beginn
- sofort
- Kontakt
-
Hojun Lim, M.Sc.
hojun.limtum.de - Ausschreibung
-
