Object-Level Sensor Fusion with Transformer Neural Networks for Object Detection in Autonomous Driving

Institut

Lehrstuhl für Fahrzeugtechnik

Typ

Semesterarbeit / Masterarbeit /

Inhalt

experimentell / theoretisch /  

Beschreibung

Successfully mastering the autonomous driving task depends highly on an accurate representation and understanding of the environment. To achieve such a detailed knowledge of the surroundings, current object detection algorithms use not just cameras but also lidar or radar data. However, utilizing the advantages of different sensor modalities without the loss of redundancy is a challenging task, which should be tackled by the development of an object-level fusion approach for camera, lidar, and radar data.

The objective of this thesis is the development of a transformer-based object-level fusion approach to combine the object detections of multiple different neural networks. The fusion network should improve the overall detection quality (especially in severe environmental conditions) while maintaining the independence of the individual detection pipelines. The goal is the development of improved object detection through sensor fusion and evaluation on real-world driving data.

The first step of this project consists of literature research on the current state of the art in object detection and sensor fusion. In the second step, a pipeline should be implemented to feed the outputs of multiple different object detection algorithms to the fusion module. In the next step, a transformer-based object-level fusion network should be developed, which improves the detection quality while preserving the independence of the detection pipelines. Finally, the results of the work should be compared to the current state of the art and validated on real-world driving data.

 

Work packages

• Literature research on object-level fusion
• Injection of the object lists into the fusion network
• Development of a transformer model for object-level fusion
• Comparison of the results to the current state of the art
• Validation and deduction of an outlook on future improvements

Voraussetzungen

Requirements

• Enthusiastic about deep neural networks
• Excited about Python programming
• Involved working attitude

 

The thesis can be written in German or English language.

 

Tags

FTM Studienarbeit, FTM AV, FTM AV Perception, FTM Informatik, FTM Fent

Möglicher Beginn

sofort

Kontakt

Felix Fent, M. Sc.
Raum: MW3508
Tel.: +49 89 289 15347
fentftm.mw.tum.de

Ausschreibung