1. BaSaMa & HiWi
    2.  

LLM-Based Grasp Point Identification for Vacuum Handling of Variable Products

Institute
Institut für Werkzeugmaschinen und Betriebswissenschaften (TUM-ED)
Type
Bachelor's Thesis / Semester Thesis / Master's Thesis /
Content
experimental / constructive /  
Description

Context

Robotic handling in disassembly and remanufacturing has to cope with high product variety at low volumes. Where a gripper in assembly handles one known part in a fixed pose, a disassembly cell sees housings, covers, and plates of different sizes, materials, and surface qualities. Vacuum grippers suit this task because they engage a single accessible surface instead of requiring form closure, but they need to know where that surface is. Grasp points are normally determined from CAD models or from grasp models learned on large sets of real objects. Neither is available for the legacy and mixed products that reach a disassembly line.

Recent work at the iwb has produced a computer-vision pipeline that converts 2D exploded-view drawings into structured component records. Each recovered component carries its callout number, its name from the bill of materials, and a segmented image of the part. Together with the maintenance manual, this amounts to a description of every part in the product, available before the product is opened, and it has not yet been used for handling.

Large language and vision-language models can reason over this kind of input. What a model proposes from documentation remains a hypothesis, because a drawing shows a silhouette rather than a surface. That hypothesis has to be checked on the physical part, which the vacuum system itself can do through its pressure signal. The thesis is therefore as much about building and testing the gripper as about the method that feeds it.

 

Objective

The objective of this thesis is to develop and demonstrate a documentation-driven approach to vacuum handling of variable products, from grasp point identification to a working pick on a UR30 robot. Specifically:

  • Review approaches for vacuum gripping and grasp point determination for variable products, and the data sources they rely on
  • Implement an already designed vacuum gripper concept for the UR30
  • Develop an LLM-based method that uses the component records from exploded-view drawings and further technical documentation to propose a grasp point with an approach direction
  • Evaluate the approach on real products, measuring how many parts receive a usable proposal and how many are picked successfully on the first attempt
Requirements
  • Interest in robotics, AI, and current trends in digitalization
  • Enthusiasm for sustainable production, disassembly, and remanufacturing
  • Self-motivation, independence, and reliability
  • Solid programming skills (Python required), experience with robot programming or ROS is a plus
  • Hands-on interest in gripper design and pneumatics, and willingness to work in the laboratory
  • Good English and German language skills
Possible start
sofort
Contact
German Bluvstein
Room: 1303
Phone: +49 89 289 15542
german.bluvsteiniwb.tum.de
Announcement