gms | German Medical Science

Introduction: During a patient's treatment a lot of information is generated. Various clinical departments and treating physicians document their actions and observations in clinical documents during the treatment process. Apart from legal issues, these documents serve as information source for other physicians that are involved in the treatment. A physician's follow-up activities depend heavily on the available information. However, identifying and processing the relevant information is very time consuming [1]. To support information retrieval and to simplify the access to the most important information, computer-based clinical decision support systems (CDSS) are developed. Beyond identifying relevant documents within a hospital information system, CDSS communicate the relevant information to the attending physician in an appropriate format or provide recommendations for treatments [2]. A major problem in obtaining the relevant information is that the majority of the medical information is recorded in unstructured, narrative format. Natural language processing methods offer a solution by structuring narrative text and identifying text passages that are relevant to a specific question or task. We introduce a semantic rule-based information extraction method for identifying adverse events that are mentioned in clinical documents. Adverse events comprise occurred complications and other unwanted events that emerged during treatment. They need to be considered by the treating physician because such events may lead to increased morbidity and mortality and thereby influence the following treatment and health care planning.

The extraction of information on adverse events from medical narratives has multiple advantages. First, the clinical decision making benefits from the availability of important, decision-relevant information. This information can then be presented to the attending physician at the right time and in a suitable form. Providing physicians with the key information instead of having to read long documents can make decision making more efficient. Second, the knowledge about adverse events is enhanced. Clinicians may learn about the likelihood of complications given certain pre-existing conditions. Thereby, adverse events can be prevented in the future. Third, the hospital management may benefit as well. Structured information allows billing systems to automatically propose encoding for occurred events (next to diagnoses and procedures), since complications and adverse happenings may influence the payment of a case. In the following sections we describe our method of extracting information on adverse events for further use in clinical decision support systems. The extracted information will be used to point out problems during therapy which then shall be considered by the attending physicians.

Methods: Whenever an unwanted event happens during a patient's treatment it needs to be documented. However, the possibilities of wording the occurrences in the text are anything but limited. To be able to automatically detect relevant passages, we manually analysed clinical documents intensively. Over one hundred annotated and anonymized medical documents in German were at our disposal, 67 surgical reports from an Ear, Nose and Throat-department (ENT), 21 surgery reports from a neurosurgical department and 25 reports from thoracic surgery. Three physicians from the respective departments tagged passages containing evidence of adverse events that from the medical point of view influence the follow-up treatment. By analysing these documents, different wordings of adverse events could be identified. Firstly, there are references to symptoms and other pathological incidences, e.g. bleeding, nausea or swelling. These references are supported by certain verbs, adjectives and adverbs that make them adverse. Secondly, there are references to procedures that failed, were problematic or not possible to begin with. In general, it needs to be considered that there are hardly any similarities in the syntactical structure of these phrases. No patterns are identifiable. Based on these results we concluded that a syntactical approach for information extraction seemed not very promising.

The first step in our approach is to semantically preprocess a given document by using the terminology server ID MACS®. It normalises words and maps them onto medical concepts of the underlying knowledge base, Wingert Nomenclature [3]. The relationship between these concepts, including the scope of negation, is depicted [4]. This created semantic structure is then in the main processing step exploited to identify concepts that represent adverse events in the text. If in a sentence a concept associated with a disease, symptom or medical procedure, respectively, is found, the surroundings are further analysed. If there are linguistic concepts to support the adversity of an event, the passage is tagged and stored in a structured form.

We conducted a study with one-on-one interviews with five physicians (two medical specialists, three assistant physicians) to evaluate the correctness of the extraction. Every participant was asked to read five surgery reports, also anonymized but not preprocessed in any form, and tag the passages that in their point of view contained adverse events. These texts were not used in the initial analysis mentioned above.

Results: We calculated the inter-annotator-agreement k to be able to claim that all participants agreed on the relevance of the passages. k resulted in 0.65, which means substantial conformity [5]. We then compared these passages with the output of our algorithm. We averaged over the five participants and calculated a recall of 0.64 and a precision of 0.78.

Discussion: We conclude that the understanding of adverse events and information needs of physicians are for most of the extracted parts the same based reflected by the conformity of their annotations in our evaluation. Therefore, we presume by extracting information about specific events during patient treatment the medical decision making can be supported. In a first evaluation, our method resulted in good precision. Nevertheless, we need to analyse the reasons for the relatively poor recall value. One possible reason might be that there are too many linguistic ways to express adversity in the texts. Considering this, our algorithm needs further refinement. When stored in a structured form, the information on adverse events can be used for different purposes. Alongside the adjustment of the algorithm for better recall, as a next step, we will include our method in an information system designed to present data from multiple sources to enhance information intake for a particular disease.