gms | German Medical Science

Introduction: For a long time there has been the problem of incorrect dosing of medications in pediatric emergencies [1], and a lot of research is being conducted into assistance services for medical professionals [2], [3]. The current solutions are all based on tablets/smartphones (e.g., [2], [3]) and thus have the limitation of not being hands-free. Therefore, there are the problems of interference with manual tasks and cross-contamination [4]. On the other hand, smartglasses are still a niche product with bad usability and limited sensor technology in a setting where usability is a key factor.

In order to minimize these issues, the interaction with smartglasses has to be kept to a minimum. Therefore, this research work has the goal to create a hands-free solution using smarglasses with additional sensors to automate important tasks (object detection, size determination and dose calculations) in a way that presents the best possible user experience.

State of the art: There are several, partly promising, research approaches to solve this problem, but they are tablet- or smartphone-based (e.g. [2], [3]). There is also research investigating the extent to which smartglasses are useful in emergency medicine, e.g. [4]. This research is mostly generic or socially centered because of the aforementioned limitations of smartglasses.

Concept: Modern existing smartglasses are extended by the necessary sensor technology, so that they are not inferior to smartphones or tablets in terms of functional scope. Therefore, the interaction is held to a minimum, so the unsatisfactory usability doesn’t influence the perceived user experience.

Implementation: The two necessary sensors (Time-of-Flight (type: VL53L1X), gyroscope/accelerometer (type: MPU6050)) are connected to an ESP32 and attached to the smartglasses via a 3D-printed mount. They continuously deliver time-stamp annotated information to a Message Queuing Telemetry Transport (MQTT) broker which is then redirected to an Apache Kafka Broker to improve the overall data process. At the same time, the smartglasses stream the camera image via Real Time Streaming Protocol (RTSP) and perform an object classification directly on the Edge Tensor Processing Unit (TPU) of the Coral USB Accelerator using ssd-MobilenetV2 [5]. If a child is detected, a box is drawn around it and the image coordinates of this box are sent to a Kafka Broker being marked with a timestamp. Here the information is merged to a final MQTT topic so that at a certain point in time the camera image corresponds to the actual sensor data. This can then be used to estimate the child’s body size.

Lessons learned: Unfortunately, there is no out-of-the-box solution to this problem. The pipeline is complex, but necessary. It can already be said that the real-time detection of children works well with the help of a TPU (approximately 20 times faster than Quad-core Cortex-A53 @ 1.5GHz). The distance sensor shows a slight blurring, as does the gyroscope/accelerometer. To what extent this is a problem for the production-ready calculations still has to be evaluated, as well as the regulatory requirements. However, the initial results are promising. Further research is needed.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.