Benchmarking Vision-Language Models for Autonomous Driving: Dataset Generation and Annotation

Institute

Professur für autonome Fahrzeugsysteme

Type

Bachelor's Thesis / Semester Thesis /

Content

experimental /  

Description

You will conduct your research at AVS!

Vision-Language Models (VLMs) like GPT-4V, Qwen3-VL, Gemini and co are increasingly being explored for autonomous driving applications. However, there is currently no systematic benchmark to evaluate whether these models actually understand vehicle dynamics and can assess trajectory quality. This is an important safety-critical capability for planning in AD.
Within this research project, you will develop the dataset and ground truth annotations for the first systematic benchmark of VLM physical reasoning capabilities in autonomous driving. You will work with several state-of-the-art datasets and models to gather the needed data.

Your Task will include:

• Processing and extracting driving scenes from the Navsim dataset
• Developing annotation pipelines for driving style classification (sporty, normal, comfortable, defensive)
• Generating ground truth labels for trajectory feasibility assessment
• Creating standardized video and image formats for VLM input
• Documenting dataset statistics and ensuring balanced scenario coverage

With your work you will actively contribute to a conference publication.

Requirements

• Python programming skills
• Familarity with data processing pipelines
• Interest in autonomous driving and machine learning 
• Basic understanding of vehicle dynamics
• Organized and independent work attitude
• Prior project work in robotics or autonomous systems is a plus
• Experience in Formula Student or similar project-based teams is a plus

Your Benefits:

• future-oriented field of research
• Yound and dynamic team
• Academic and professional support
• Organized and structured project
• Direct contribution to a scientific publication aimed to be published at a top robotics conference
• Project work in english or german
• with data processing pipelines
• Interest in autonomous driving and machine learning 
• Basic understanding of vehicle dynamics
• Organized and independent work attitude
• Prior project work in robotics or autonomous systems is a plus
• Experience in Formula Student or similar project-based teams is a plus

Your Benefits:

• future-oriented field of research
• Young and dynamic team
• Academic and professional support
• Organized and structured project
• Direct contribution to a scientific publication aimed to be published at a top robotics conference
• Project work in english or german

Tags

AVS Schaefer

Possible start

sofort

Contact

Finn Rasmus Schäfer
finn.schaefertum.de