Siamese Network for assembly step recognition and quality assessment for Human-Robot Collaboration

Interest in Human-Robot Collaboration is continuously growing in the industrial field, leveraging human cognitive capabilities with the consistent repeatability and reliability of robots to enable flexible production while enhancing working conditions. Within this context, the goal of this paper is to recognize assembly steps by presenting a Siamese Network-based approach adaptable to different assembly scenarios. The Siamese Network compares images captured during the process with a previously acquired reference dataset to identify the last executed assembly step and evaluate its correctness. The model’s effectiveness was proved offline by comparing its performance with pre-existing OpenCV feature-matching tools. Additional online experiments were performed on three assembly cases exploiting a hand-tracking algorithm to communicate the operation’s completion. The results highlight the versatility and reliability of the proposed methodology, which offers a straightforward solution for recognizing assembly steps without the need for extensive training or complicated setups.