gms | German Medical Science

Objective: Hand tracking is a challenging problem that recently gained relevance with the development of cheap consumer-level depth cameras and virtual reality devices. We present a framework for dynamic evaluation of the movements of flexion, extension, abduction and aduction for patients with rheumatoid arthritis.

Materials and Methods: This framework estimates angle measurements from joints computed by a hand pose estimation algorithm using a depth sensor (Realsense SR300). Given depth maps, our framework uses Pose-REN (Guo et al., 2018), which is a state-of-art hand pose estimation method that estimates 3D hand joint positions using a deep convolutional neural network. Pose-REN was trained on the BigHand2.2M dataset (Yuan et al., 2017), which was built using active hand movements of healthy subjects. The pose estimation algorithm runs in real-time, allowing users to visualise 3D skeleton tracking results at the same time as the depth images are acquired. Once 3D joint poses are obtained, our framework estimates a plane containing the wrist and MCP joints and measures flexion/extension and abduction/aduction angles by applying computational geometry operations with respect to this plane.

Results: We analysed basic flexion and abduction movement patterns, extracting the movement trajectories. Our preliminary results show that by comparing these trajectories, it is possible to discriminate patients with AR from healthy patients. The angle between joints can be used as an indicative of the subject's current movement capabilities. Although the measurements are not as accurate as those btained through with goniometry, acquisition is much easier. The system can be used with and without orthosis.

Conclusions: We obtained promising results on the assessment of hand movement for occupational therapy using computer vision. Our framework allows the acquisition of measurements with minimal intervention and significantly reduces the time this task takes.