Segmentation-Based Approach for a Heuristic Grasping Procedure in Multi-Object Scenes

Object manipulation in unstructured environments is important for many industrial applications where the items vary in shape, size, and material. This paper introduces a two-step pipeline for object picking, which combines instance segmentation with a heuristic based grasp point selection. The grasping points are determined using the 2D segmentation masks and depth images. A voxel-downsampling procedure reduces the depth noise, and the Theil-Sen algorithm ensures a robust linear regression for the grasping attitude determination. Unlike other methods, our approach does not require extensive training, as well as a fine labelled dataset for picking, and hence it is also independent of object shapes. Using SAM's ViT-h version and a binary object detector trained on a large dataset, our method is robust and class agnostic. The experiments, made using a RealSense D435i camera and a Racer 3 manipulator, show that our pipeline has a good success rate in simple and moderately complex scenarios, balancing computational efficiency and accuracy.