Uncertainty-aware human pose estimation for collaborative aerial robots

Human pose estimation (HPE) algorithms based on deep learning allow for extraction of pose information (skeletal keypoints) from images or videos. While the pixels-to-keypoint correspondence in images may be determined with reasonable confidence and performance, applications of HPE algorithms to robotics are much more challenging, because they require estimation of human poses in three-dimensional coordinates (3D HPE). Moreover, collaborative robots using HPE for localizing humans in the workspace may need a measure of the quality of the estimation, to avoid taking decisions based on wrong detections of humans in the camera data. This experimental thesis focuses on the design and implementation of a “Quality of Estimation” (QoE) heuristic, giving a measure of the uncertainty in the estimation of the 3D human pose. The QoE heuristic is calculated using heatmaps from the pose estimator and simple geometric priors on the shape of the human body, and it can be computed in real time from a live camera feed. Considering a human-collaborative UAV handover setting, the proposed heuristic has been validated on a real scenario including a stereo camera and a hexacopter, showing a favorable performance to power ratio. Moreover, the heuristic was used in a simulated and simplified human-robot collaboration setting to develop exploration policies based on reinforcement learning. Experimental results show that the heuristic allows to learn an exploration policy minimizing the uncertainty on the estimated 3D human pose. All the software was developed and tested on an actual embedded platform (Nvidia Jetson TX2) and a hexacopter equipped with a RGB-D camera from the University of Twente.