gms | German Medical Science

Background: Communicatively or cognitively impaired individuals may not be able to express the presence of pain adequately, leading to potential underrecognition and undertreatment of pain [1], [2]. Examination of facial expressions has proven to be a valuable objective method to infer the existence of pain [3], but in a clinical setting this may not be convenient for medical staff, because of complexity and time consumption of the task. Automated pain detection can solve these issues. The objective of this work was to evaluate the role Convolutional Neural Networks (CNNs) can play in the development of an image-based method for pain detection.

Methods: The dataset utilized for training and testing CNN models was the UNBC-McMaster Shoulder Pain Expression Archive Database [4]. It contains 48,398 images, each of which depicts one of 25individuals, suffering from chronic shoulder pain. Each image was coded by certified FACS [4] coders and PSPI [4] (pain) levels were calculated subsequently. As the dataset was highly imbalanced, PSPI levels were merged into three classes, a no-pain class, mild-pain class and notable-pain class. The dataset was then split into a training- and testset, by using all images depicting one specific individual for the testset and the remaining for training. This approach enables the evaluation of models, in terms of ability for pain detection for individuals not contained in the training set and provides more data for training. To reduce irrelevant information from the images, everything surrounding an individual's face was cropped out using the Multi Task Cascaded Convolutional Neural Network [5]. To increase the size of the dataset, images were augmented (horizontal flipping, rotation and change of brightness, contrast, saturation and hue). Five CNNs were trained for the task. Two CNNs were based on the AlexNet [6] architecture and three were based on the ResNet [7] architecture. Transfer learning approaches were also used (ImageNet [8] and VGGFace2 [9]).

Results: All models had considerable difficulties distinguishing between no-pain and mild-pain. For the notable-pain class the ResNet-50-ImageNet CNN model achieved promising results for clinical application with a good performance in recall (0.81) and arguably acceptable results for precision (0.6), accumulating in a F1 score of 0.69. ResNet-50-VGGFace2 and AlexNet-plain (plain: no transfer learning) came in close with F1 scores of 0.66 and 0.59, but did not perform well on recall (both 0.51) (precision 0.93 and 0.7). ResNet-50-plain scored far worse with a F1 score of 0.5 (recall 0.38; precision 0.73) and AlexNet-ImageNet scored lowest by completely failing to detect signs of notable pain (F1 score 0.17; recall 0.09; precision 1).

Conclusion: The results show that promising results can be obtained even though only a small amount of data was available for training. Therefore, it can be concluded that CNNs do have the potential to contribute significantly to facial expression based automated pain detection. The models may be improvable by extending transfer learning approaches, cropping out even more of the background and preserving only information contained in the oval shape of the face. It should also be experimented with aligning all faces in the training- and testset.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.