[SA/IDP] AI Agents for Operating and Managing an Autonomous Driving Software Stack

Institute

Lehrstuhl für Fahrzeugtechnik (TUM-ED)

Type

Semester Thesis /

Content

experimental / constructive /  

Description

Motivation

Operating an autonomous driving research vehicle involves numerous manual and repetitive tasks: mission goals must be set by hand, all software and hardware components must be verified before each test drive, and recorded data must be sorted, uploaded, and documented afterwards. These tasks are time- and resource-consuming, error-prone. Recent advances in large language model (LLM)-based AI agents open up the possibility of delegating such tasks to an intelligent assistant that can perceive the system state, reason about it, and act through well-defined tool interfaces. In the light of this, an AI agent integrated into an autonomous driving software stack such as Autoware could set goal poses from natural language commands, perform automated start-up checks of all necessary components, and manage recorded driving data, including upload and automatic summarization of each recording session. 

Thesis Topic

This thesis focuses on developing and integrating an LLM-based AI agent into the Autoware software stack of our research vehicle, with an aim on mission assistance, automated system checks, and data management as the target applications.

The thesis can be divided into the following tasks:

• Literature research on current methods for LLM-based AI agents, tool use, and their application in robotics and autonomous driving
• Design of an agent architecture and its tool interfaces to the Autoware stack (e.g., via ROS 2 and Zenoh bridge)
• Implementation of a Proof-of-Concept (PoC) covering selected use cases: 
  • natural-language mission commands (e.g., goal pose setting), 
  • automated start-up check of necessary components, 
  • management and summarization of recorded driving data
• Evaluation of the implemented agent regarding reliability, task success rate, and usability on the research vehicle or in simulation
• Documentation of the work

Requirements

• Proficient in Python and experience with ROS2
• Working knowledge of distributed computing systems
• Self-reliant working style and motivation
• Good communication skills in English or German
• (Good to have) experience with LLM deployment frameworks (e.g., vLLM or ollama) and/or LLM API calls
• (Good to have) experience with skill prompting and agent harnessing

 

Tags

FTM AV, FTM AV Safe Operation, FTM Su, FTM Informatik

Possible start

sofort

Contact

Xiyan Su, M.Sc.
Room: MW3507
Phone: +49 89 289 15340
xiyan.sutum.de

Announcement