Photorealistic off-road digital twins: Simulating dynamic objects, changing lighting, and altered terrain with 3D Gaussian Splatting

Institut

Lehrstuhl für Fahrzeugtechnik (TUM-ED)

Typ

Semesterarbeit / Masterarbeit /

Inhalt

experimentell / konstruktiv /  

Beschreibung

Autonomous off-road driving (e.g., in agriculture, forestry or disaster response) must cope with unstructured terrain such as grass, gravel and mud, whose appearance varies strongly with season, weather and lighting. Real-world tests are costly and hard to reproduce, and classical simulators rarely capture this diversity, resulting in a large sim-to-real gap.
3D Gaussian Splatting (3DGS) is the state of the art for real-time, photorealistic scene reconstruction. Yet existing 3DGS simulators mostly target on-road scenes and only replay recorded drives instead of varying them, e.g., with other road users, different lighting or altered terrain.
In this project, we build an off-road simulation from real vehicle sensor data in which reconstructed scenes can not only be replayed but also realistically edited and varied. The thesis can be carried out independently or as part of a team.

 

Your Contribution
You take ownership of one research area, for example:

• Dynamic scenes: moving vehicles, people and swaying vegetation
• Relighting and weather: time of day, wet or dusty conditions
• Scene editing: inserting obstacles, altering terrain such as ruts or puddles
• Generative priors: diffusion-aided  3DGS

In your area, you review the state of the art, implement your method in the shared pipeline, and evaluate it with real vehicle data.

Voraussetzungen

What should you bring?
• Very good programming knowledge in Python (PyTorch); C++ is a plus
• Experience with AI-Tools to accelerate and improve development
• Solid background in computer vision, 3D geometry or machine learning
• High drive, self-motivation and an independent, structured way of working
• Interest in off-road autonomy, 3D reconstruction and simulation

 

✉ richard.taupitz@tum.de

Möglicher Beginn

sofort

Kontakt

Richard Taupitz, M.Sc
Raum: MW 3507
Tel.: +49 89 289 10499
richard.taupitztum.de

Ausschreibung