gms | German Medical Science

Objectives: In previous research projects, an integrated operation theatre has been realized, based on the open communication standard IEEE 11073-SDC. This allows for the interaction with connected medical devices via centralized workstations. Generally, the sterility of an interaction with a user interface during surgical interventions is critical, preventing the surgeon from adjusting views of intraoperative images or complex configurations personally. Hence, the goal of this work is to assess the integration of a gesture control system (GCS) within a surgicalworkstation and identify requirements and constraints.

Methods: A requirement analysis has been conducted, starting with literature research regarding similar use cases and implementations of GCS in the surgical field. Furthermore, field observations during orthopaedic and neurosurgical operations have been carried out. Experienced surgeons have been questioned regarding their demands for a GCS in the OR and possible control gestures within interviews and by means of an anonymous survey conducted with the help of the DGOU. After analyzing the requirements, a first functional model of the GCS has been integrated into the workstation. It uses two different camera systems to track gestures and translates them into cursor movements on the surgical workstation. The far range is monitored by the Kinect Version 2 by the Microsoft Corporation, the close range is realized by the Leap Motion Controller by Leap Motion Inc.

Results and conclusion: According to the results of the interviews and the survey, there are special cases, in which the surgeon prefers to interact directly with devices instead of relying on the unsterile OR nurse. A GCS is perceived as a suitable way of interaction and as an added value for an integrated OR-System. Test subjects preferred to utilize the same set of gestures for both gesture recognition devices, conveniently switching between them. A set of gestures mimicking touchscreen interaction appeared suitable, utilizing already known gestures and corresponding system responses. Moreover, gesture control should be able while holding a surgical instrument with the active hand. To reduce the risk of being unsterile, the gestures should be performed in front of the surgeon's chest. For the long range GRS it has been found, that the line of sight problematic between camera and surgeon may have an impact on the workflow in the OR, which leads to further limitation of the already-restricted space for the OR personnel.

First evaluation results seem to be promising. Surgeons have been able to interact naturally with the GCS after a short learning phase. The surgeon is now able to interact with the surgical workstation without touching an unsterile surface. In the future, the impact of the GCS on usual operation procedures have to be reduced. Additionally, the combination of speech and gesture control has to be investigated in matters of the enhancement of usability and safety.