Hardware-Aware Deployment of VLA Foundation Models on Edge Compute

Institut

Professur für autonome Fahrzeugsysteme (TUM-ED)

Typ

Masterarbeit /

Inhalt

experimentell / theoretisch / konstruktiv /  

Beschreibung

Background

Join our team to solve the critical latency bottleneck in modern robotics by optimizing massive Vision-Language-Action (VLA) models for real-time edge deployment! Are you eager to bridge the gap between heavy cloud-based AI and real-time physical systems? Do you want to work at the forefront of efficient AI, where optimizing memory and inference speed directly dictates whether a robot succeeds or fails? This project offers a unique opportunity to deploy state-of-the-art foundation models directly onto our robots' onboard compute. Training a massive VLA model is only half the battle. To perform highly dynamic tasks, robots like the Unitree G1 require control loops running at 50Hz or higher. Deploying multi-billion parameter models directly onto edge devices like the NVIDIA Jetson Thor without severe latency is our goal. You will take trained VLA models and optimize them for onboard execution. We will utilize quantization, model pruning, and TensorRT compilation. We will design hardware-aware inference pipelines that balance the model's accuracy with the strict real-time constraints required for stable robotic control, ensuring the "brain" reacts fast enough to control the "body."

Example Thesis Topics (subject to availability):

• Quantization for VLA Edge Inference: Investigate the impact of INT8, FP8, and mixed-precision quantization on the action output accuracy of VLA models deployed on the NVIDIA Jetson Thor.
• TensorRT Optimization for Flow-Matching Action Generation: Optimize complex diffusion or flow-matching action generation heads within VLA models to meet strict 50Hz inference requirements on edge hardware.
• Asynchronous Inference Architectures for VLA Control: Design a deployment architecture that decouples the heavy, slow vision/language processing from the fast, high-frequency low-level action generation, ensuring the robot's control loop never drops a frame.
• Benchmarking Foundation Models on Next-Gen Robotic Edge Compute: Conduct comprehensive profiling of memory bandwidth, thermal throttling, and compute utilization of various VLA architectures running natively on the Jetson Thor.

Technologies Used Python, C++, PyTorch, TensorRT, ONNX, CUDA, NVIDIA Jetson Thor, Edge AI, Model Quantization, Efficient Inference, Real-time Systems, ROS 2.

 

Your Benefits: Join a High-Performance Robotics Team

• Impactful Research: Work on a project where your code doesn't live in a silo; it is a critical gear in an end-to-end pipeline. Your results will directly enable robots to perform complex tasks.
• Top-Tier Hardware Stack: Gain exclusive hands-on experience with NVIDIA DGX (training), Jetson Thor (inference), and Unitree Humanoids/Quadrupeds - very similar stack used by industry leaders like Tesla, Figure AI, and Physical Intelligence.
• Scientific Publication: We aim for high-impact results. If your work meets the quality standards, we will co-author and submit a paper to top-tier robotics/AI conferences (e.g., ICRA, IROS, CoRL, or CVPR).
• Professional Career Launchpad: This thesis is designed to mirror the workflow of elite AI labs. We provide dedicated mentorship and professional support to help you land roles at top-tier robotics startups or Big Tech AI labs.
• Dynamic Lab Culture: You will be part of a "squad" of motivated Master’s students working in parallel, fostering a collaborative, fast-paced, and supportive environment.

 

Requirements

We are looking for students who know their thesis is not just as a degree requirement, but as a career-defining project.

Must-Have:

• English Proficiency: High level of written and spoken English (the language of our research and documentation).
• Proactive Mindset: You are comfortable with a "fail fast, learn fast" approach and is comfortable solving hands-on hardware/software integration challenges.
• Independence: Ability to own a technical module and drive it forward while communicating effectively with the rest of the team.
• Growth Path: A passion for Robotics/AI and an eagerness to learn new technologies.

Nice-to-Have (The "Plus"):

• Technical Foundation: Proficiency in Python and/or C++.
• Domain Experience: Prior exposure to PyTorch, ROS 2, or physics simulators (Isaac Sim/MuJoCo).

• Hardware Skills: Experience working with robotic hardware, sensors, or VR systems.

   

Ready to build the future of Embodied AI? Send your CV, recent transcript, and a brief email on why you are the right fit for this specific "squad" and your career goals.

Möglicher Beginn

sofort

Kontakt

Roberto Brusnicki
roberto.brusnickitum.de