gms | German Medical Science

Objective: There have been several attempts to supply surgeons with intraoperative 3D information of planning data overlaid with the surgical field. So far none of these solutions is clinically used. The aim of the ARNO-System (Augmented Reality for Navigated Operating Microscopes) is to provide the surgeon with a stereoscopic full colour three dimensional overlay of preoperatively planned models.

Methods: A concept to allow the overlay of virtual 3D objects of interest derived from the patients images onto the surgical field was developed. In order to achieve a correct correlation between the position and orientation of the model according to the anatomy, the microscope optics and image overlay source have to be calibrated. The presentation of the model has to be prepared stereoscopically and injected into the eyepieces of the microscope to be overlaid onto the surgical field. For image injection, an image source as well as an optical solution had to be developed.

Results: Stereoscopic visualization of 3D objects is achieved using the VTK-software (visualization toolkit) by definition of a rigid frame of two virtual VTK-cameras within the microscope optics. The virtual optical axis of visualization was readjusted according to the reduced parallaxes of the microscope compared to the surgeon’s eyes. To deliver the images into the microscope beam, a microoptical bank was constructed and coupled with the beam-splitters used for photo and the assistants eyepiece. Because the virtual cameras are ideal according to lens-distortions, no calibration of the cameras is necessary but the displays have to be calibrated. Manual calibration to achieve the position of each virtual camera resulted in an average display deviation of 0,04 mm and a maximum of 0,15 mm. These parameters as well as current zoom and focus have to be applied to the virtual camera objects. First tests showed good results regarding accuracy using a phantom brain with built-in tumours and a stereoscopic overlay of 3D models. With daylight-projectors replacing the initially used microdisplays, the objects could be recognised with good contrast even with the microscope light at maximum intensity.

Conclusions: Stereoscopic full colour augmented reality is possible by using the ARNO-System. With the navigated and scaled overlay of virtual 3D-objects onto the surgical field, a more intuitive orientation for the surgeon will be possible without interruption of the surgical workflow. In respect to the contrast rich surgical field, the display of the models still has to be optimised along with first clinical tests.