From Perception to Grasp: a Real-Time Framework for Object-Agnostic Human-to-Robot Handovers

The handover of objects consists of a joint action of a giver transferring an object to a receiver. Providing such capability to collaborative robots is crucial for human-robot collaboration in manufacturing scenarios, especially when people are involved. The objective of this work is to provide a general framework for a real-time, object-agnostic human-to-robot handover. After the generation of a point cloud of the scene by means of a single RGB-D sensor mounted on the robot's gripper, the proposed method comprises three phases. The first one involves the segmentation of the scene, which allows to distinguish between the human hand and the object being held. The second phase consists of generating a set of possible grasps from the elaborated scene. Finally, the third phase regards the selection of the feasible grasp poses according to three defined criteria. The proposed approach consists of constructing a possible implementation for the Human-To-Robot handover task, leveraging and modifying models found in the literature. In particular, a faster version of the EgoHOS model, Fast-EgoHOS, for the segmentation task has been proposed and compared to its original implementation, named Complete-EgoHOS. Moreover, a comparison has been conducted between three state-of-the-art models for grasp generation, which are GraspNet, 6-DoF GraspNet, and Contact-GraspNet. Based on the best segmentation-grasp detection model configuration, identified through an offline evaluation in terms of accuracy, execution time, and grasp quality, an online evaluation is performed. This is conducted by measuring the success rate and the time needed to perform an handover attempt. The presented approach allows us to achieve a value of IoU of Fast-EgoHOS of 78.8% compared to the 82.84% of Complete-EgoHOS, with a significant advantage in terms of inference time. In the online evaluation, a grasp success rate of 82.9% is achieved with the Fast EgoHOS-Contact-Graspnet configuration and of 80.3% with Fast-EgoHOS-Graspnet. Results are obtained considering a set of 19 distinct objects presented in various positions.