TY - GEN
T1 - Augmented Reality in Robotic Mitral Valve Repair
AU - Devarmani, Darshan
AU - Malmivirta, Ilona
AU - Lasa-Rivas, Alejandro
AU - Brembilla, Valentina
AU - Grandvallet, Christelle
AU - Giroletti, Laura
AU - Graniero, Ascanio
AU - Agnino, Alfonso
AU - Lanzarone, Ettore
AU - Thomann, Guillaume
PY - 2025/10/1
Y1 - 2025/10/1
N2 - Robotic-assisted cardiac procedures, especially mitral valve repair, benefit significantly from robotic devices, although positioning robotic ports in the chest currently depends solely on surgeon expertise. Augmented reality (AR) technologies have evolved to provide a composite view of the patient in the operating room with co-located virtual holographic representations of medical data, creating a more immersive surgical experience. This study aims to develop a preliminary AR prototype for testing in a hospital in Bergamo, Italy. From February to June 2024, an AR tool was developed to integrate AR HoloLens2® glasses with the Da Vinci® surgical robot, enabling surgeons to use CT scan holograms overlaid on patients for precise port positioning. The development focused on creating a Unity software integration and a custom 3D model of the patient’s heart. Using Slicer3D® software, a pipeline was developed to create a 3D model from pre-surgical CT scans, which was successfully integrated into Unity for visualization on the patient’s chest. The study demonstrated the feasibility of integrating AR into robotic-assisted surgeries, effectively combining technologies like DICOM, Unity, and Slicer for enhanced surgical workflows.
AB - Robotic-assisted cardiac procedures, especially mitral valve repair, benefit significantly from robotic devices, although positioning robotic ports in the chest currently depends solely on surgeon expertise. Augmented reality (AR) technologies have evolved to provide a composite view of the patient in the operating room with co-located virtual holographic representations of medical data, creating a more immersive surgical experience. This study aims to develop a preliminary AR prototype for testing in a hospital in Bergamo, Italy. From February to June 2024, an AR tool was developed to integrate AR HoloLens2® glasses with the Da Vinci® surgical robot, enabling surgeons to use CT scan holograms overlaid on patients for precise port positioning. The development focused on creating a Unity software integration and a custom 3D model of the patient’s heart. Using Slicer3D® software, a pipeline was developed to create a 3D model from pre-surgical CT scans, which was successfully integrated into Unity for visualization on the patient’s chest. The study demonstrated the feasibility of integrating AR into robotic-assisted surgeries, effectively combining technologies like DICOM, Unity, and Slicer for enhanced surgical workflows.
KW - Augmented Reality
KW - Mitral Valve Repair
KW - Robotic-Assisted Mitral Valve Surgery (RAMVS)
KW - Surgical Workflows
UR - https://www.scopus.com/pages/publications/105018912013
U2 - 10.1007/978-3-031-97781-7_24
DO - 10.1007/978-3-031-97781-7_24
M3 - Chapter in a published conference proceeding
AN - SCOPUS:105018912013
SN - 9783031977800
T3 - Lecture Notes in Computer Science
SP - 328
EP - 345
BT - Extended Reality - International Conference, XR Salento 2025, Proceedings
A2 - De Paolis, Lucio Tommaso
A2 - Arpaia, Pasquale
A2 - Sacco, Marco
PB - Springer
CY - Cham, Switzerland
T2 - International Conference on Extended Reality, XR Salento 2025
Y2 - 17 June 2025 through 20 June 2025
ER -