gms | German Medical Science

Introduction: Free-text fields in medical protocols often contain important information expressed in many variations. Thus, the content analysis of these free-text fields is highly important [1]. But the analysis is often associated with a high level of manual effort because they are not standardized or structured [2]. In the project „ILEG – Inanspruchnahme, Leistungen und Effekte des Gemeindenotfallsanitäters (Utilization, services and effects of the community emergency paramedic) protocols comprising free-texts were assessed [3]. For easier statistical analysis, the content of the free-text fields should be assigned to pre-defined categories. We aimed at developing and evaluating a flexible automatic approach for content analysis of those free-text fields without using commercial, costly software. This results in following research questions:

1.

What are the requirements for an approach for automated analysis of free-text fields?

2.

Is it possible to develop an easy-to-use, cost-effective, low-barrier tool with well-known techniques?

3.

Are the results generated automatically by the tool comparable to manual free-text field analysis by experts?

Methods: To develop the requirements we used (1) brainstorming and (2) think-aloud while two researchers analysed free-text fields of protocols of a pilot study [4].

In compliance with the founded requirements a routine was developed using Visual Basic for Applications (Excel) version 7.1. Evaluation followed by checking manually the result of the automated assignment by repeated test coding of free-text fields of a pilot study.

By a descriptive analysis we compared the results of our routine to the results of a manual assessment of three researchers.

Results: For each pre-defined category, keywords in different formulations, with possible spelling errors and also different abbreviations has to be declared. We identified more requirements during tool evaluation: exception rules for keywords has to be declared and it must be able to realise that the content of a text field has been assigned to a category only once.

The routine runs through three loops, first the categories and second all keywords for each category are loaded in sequence. In the last loop, the current keyword is searched for in the free-text fields. If there is a hit, the content is assigned to the currently loaded category.

Comparison of automated analysis to the manual approach resulted in 76 of 89 (85.4%) free-text fields assigned to the same category. Comparing the results of the manual assessments revealing high biases in the individual decisions. Only 13 of 89 (14.6%) free-text fields were categorised the same by all researchers.

Discussion: Using Excel for content analysing in empirical social research is already described - but not for automatic analysis [5]. We are aware that more complicated and interacting rules are required for more comprehensive analyses, e. g. in media research [6]. We also refrained from evaluating and weighting words and terms [7].

Our meaning is that one of the most important factors of different manually assignment is the different special training of the researchers.

Conclusions: The analysis of short free-text fields with a VBA program is possible and a cost-effective, low-barrier alternative to commercial, often inflexible, tools.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.