Computer Vision-Based Anatomical Structure Recognition in Da Vinci Robot-Assisted Radical Prostatectomy

Robot-assisted radical prostatectomy (RARP) is a complex surgical procedure in which the identification of anatomical structures is essential for understanding the surgical scene. In procedures performed using the da Vinci Surgical System, computer vision and deep learning techniques may provide additional information from the endoscopic video stream, supporting surgical training and, potentially, real-time assistance. This thesis investigates YOLO-based instance segmentation for anatomical structure recognition in RARP videos. The experimental analysis initially focused on seminal vesicle segmentation and was subsequently extended to prostate segmentation. A dedicated desktop application was developed to support the workflow, including video management, manual and automatic frame selection, annotation, model-assisted review, training, and evaluation. Given the limited availability of annotated surgical data, particular attention was devoted to dataset characterization and data augmentation. Images from different surgeons and procedures were analysed in terms of visual and geometrical characteristics, which were used to define controlled augmentation strategies while preserving realistic surgical appearance. Models were evaluated both within individual surgeon datasets and under cross-surgeon conditions to investigate their generalization to unseen surgical data. The experiments showed better overall performance for prostate segmentation than for seminal vesicle segmentation, consistently with its larger extent and generally clearer representation in the surgical field. Cross-surgeon evaluation showed a reduction in performance, highlighting the variability between surgical procedures. Different fine-tuning strategies were therefore evaluated to investigate whether a previously trained model could incorporate a limited amount of newly annotated data and be adapted between surgeons while preserving generalization to subsequent procedures. Overall, this work presents a data-oriented framework to support anatomical structure recognition in robot-assisted surgery. The developed workflow is intended to facilitate future research in surgical image segmentation, providing an initial methodological and software support for video management, frame selection, annotation, and model development. Although developed and evaluated in the context of RARP, it provides a flexible basis that can be extended to other anatomical structures and segmentation tasks.