LLM-driven Vulnerability Discovery and Adversarial Scenario Generation

Institut

Professur für autonome Fahrzeugsysteme

Typ

Semesterarbeit / Masterarbeit /

Inhalt

experimentell / theoretisch / konstruktiv /  

Beschreibung

Join us in pioneering a new era of autonomous vehicle safety by leveraging Large Language Models (LLMs) to automatically design reward functions and generate diverse, effective adversarial driving scenarios!

Are you passionate about making AI systems robust against unforeseen challenges and enhancing safety through intelligent testing? Do you want to work at the intersection of advanced AI, reinforcement learning, and safety engineering for autonomous vehicles? This project offers a unique opportunity to build an automated, intelligent vulnerability discovery pipeline.

Current methods for discovering weaknesses in autonomous driving policies often struggle to find diverse and truly effective vulnerabilities. LLMs, however, possess a profound capability to act as meta-programmers or domain experts for safety. They can automatically design reward functions for Reinforcement Learning (RL) training, guiding RL agents to expose specific system weaknesses. This means LLMs can learn to identify and formalize "unsafe" conditions from high-level descriptions or even human feedback, translating abstract safety principles into executable reward signals. This bridges the critical gap between human safety intuition and algorithmic optimization, offering a highly scalable approach to safety assurance.

You will develop a framework where an LLM (or a VLM with strong reasoning capabilities) acts as an "adversarial expert." This expert will generate detailed descriptions of challenging scenarios or directly design sophisticated reward functions for an RL agent. The RL agent, guided by these LLM-designed rewards, will then learn to expose specific vulnerabilities in the autonomous driving system. Your research could specifically focus on generating "socially adversarial" scenarios that exploit human-like decision-making patterns or ethical dilemmas, pushing the boundaries of traditional safety testing beyond simple physical collisions. This approach fundamentally shifts safety testing from a manual, labor-intensive process to an automated, intelligent vulnerability discovery pipeline, promising to accelerate the development of more robust and reliable autonomous systems.

---

Example Thesis Topics

• LLM-Guided Reward Function Design for Adversarial RL: Develop a framework where an LLM generates structured reward functions for an RL agent, enabling it to learn behaviors that induce specific failure modes in an autonomous driving policy (e.g., causing the AV to perform an unsafe maneuver).

• Natural Language-to-Scenario Generation for Vulnerability Testing: Implement a system where an LLM takes high-level natural language descriptions of problematic driving situations (e.g., "a pedestrian suddenly darts out from behind a bus") and generates detailed, executable adversarial scenarios for simulation.

• Social Adversarial Attacks using LLM-driven Agents: Design LLM-controlled or LLM-informed adversarial agents (e.g., other vehicles, pedestrians) within a simulation environment that learn to interact with the autonomous vehicle in ways that expose its limitations in social reasoning or ethical decision-making.

• Automated Safety Property Formalization with LLMs: Investigate how LLMs can be used to formalize abstract safety properties (e.g., "always yield to emergency vehicles," "maintain safe following distance") into verifiable logical constraints or reward signals for safety testing.

• Human-in-the-Loop Feedback for LLM-driven Vulnerability Discovery: Develop a system where human safety experts provide natural language feedback on identified vulnerabilities, and an LLM refines its scenario generation or reward function design based on this feedback, creating a continuous improvement loop.

 

Technologies Used

Python, PyTorch/TensorFlow, Large Language Models (LLMs), Vision-Language Models (VLMs), Reinforcement Learning (RL) frameworks (e.g., Stable Baselines, Ray RLlib), Autonomous Driving Simulators (e.g., CARLA, Scenic, SUMO), Adversarial Examples, Scenario Generation, Safety Verification, Deep Learning, Natural Language Processing, Reward Function Design, Multi-Agent Systems.

Voraussetzungen

We're looking for students with a strong background in deep learning, reinforcement learning, and a passion for making AI systems safe and robust.

• Solid understanding of deep learning frameworks (PyTorch/TensorFlow).

• Experience with Large Language Models (LLMs) and their applications.

• Strong foundation in Reinforcement Learning (RL) algorithms.

• Familiarity with autonomous driving simulation environments (e.g., CARLA, Scenic).

• Proficiency in Python.

• Motivation to work on safety-critical, high-impact research.

 

If you're ready to make a tangible impact on the future of autonomous vehicles, send us an initiative application.

Please include:

• A short motivation letter highlighting your interest in LLMs, reinforcement learning, and autonomous vehicle safety.

• Your CV.

• A recent transcript of records.

• (Optional) Any relevant project work or code samples demonstrating your experience in relevant fields.

Möglicher Beginn

sofort

Kontakt

Roberto Brusnicki
roberto.brusnickitum.de