gms | German Medical Science

Using claims data for research has great potential, because these data include large numbers of observations and have a high reliability [1], [2], [3], [4]. Therefore, such data are used, for example, to analyze treatment provision [5], [6] or health service utilization [7]. Using claims data for research is well established in the United States [8], [9] and in single European countries [10], [11]. However, multi-national approaches scarce.

A novel approach aimed to use claims data on European level for dental care research, by assessing the quality of processes of dental health care services and to optimize oral health towards prevention. The EU-funded project ADVOCATE (“Added Value for Oral Care”) analyzes claims data from health insurance providers, health funds and health authorities in six European countries: Denmark, Germany, Hungary, Ireland, the Netherlands, and the United Kingdom [12]. The data were acquired from the data owners into a central data repository. Oral health measures for dental care were defined by a group of experts and were analyzed to determine which national characteristics had a positive influence on these indicators [13].

The measures refer to topics regarding the access to dental care, symptoms and diagnosis, health behaviors, oral prevention and patient perception. Examples for oral health measures are the number of specific treatments, for example periodontal examination, root canal treatment and tooth extraction in a specific time period. The aim was and is to recommend successful strategies to other countries.

Claims data are stored in databases of a data owner and specific data sets had to be generated for the ADVOCATE project. To what extent and under which conditions the data can be used for research is determined by each data owner [14]. The conditions refer both to country-specific data protection and privacy acts and internal policies [15], [16]. The variety of characteristics in these regulations require different approaches to access, transfer, and storage of the data.

In this paper, the generic steps, which are necessary to establish an individual data acquisition process, are described and formalized in a process model, to provide researchers with a methodological framework. Thus, the aim of this paper is to describe a standardized approach for acquiring international administrative routine data for research, based on the experiences gained during the ADVOCATE project.

A data acquisition process model was developed in an evolutionary process by adapting and tailoring the methodology of agile modeling [17]. The process followed the guidelines and recommendations of the “Good Practice of Secondary Data Analysis” (GPS), which provides guiding principles for conducting studies based on secondary data analyses [18]. GPS was systematically analyzed and necessary activities were determined. Information and data flow were formalized according to the Business Process Modeling Notation (BPMN) [19] using Microsoft Visio 2013 [20].

The next step was to determine specific activities and implementations of the general process for the ADVOCATE project. Therefore, a qualitative content analysis of data usage agreements (DUAs), which were consented with the data owners, was performed [21]. A coding scheme for data usage, data protection, and privacy aspects was developed. The code structure was developed in a mixed deductive-inductive approach [22]. Deductively created codes represent general aspects of data access, transfer and usage. Inductively created codes represent implementations in ADVOCATE. The relevant text passages in the DUAs that contained statements on specific implementations for the data acquisition process were then identified. The DUAs were examined and systematically coded by three independent reviewers using the software package MAXQDA 11 [23].

GPS was analyzed with a focus on practical aspects for the data acquisition process. Because GPS contains recommendations not only for acquiring secondary data, not all of the recommendations were implemented and certain activities were either added or were deemed to be irrelevant. Therefore, all guidelines were included, except for those, which focused on the planning of the study and which are not related to data acquisition (guideline 1, 2, 4). Furthermore, guidelines that are related to later stages than data acquisition (guideline 7, 10, 11) were excluded. The recommended plausibility checks in guideline 6.4 were extended towards the three-level quality and validity audit according to Horenkamp-Sonntag [24].

The analysis of GPS resulted in a four-phase data acquisition process that describes the general workflow: First, data access and usage are prepared, which involves the formal definition of general conditions. These conditions are subsequently tested by sharing a data sample. A quality and validity audit is performed next to check data quality of the data sample and the compliance of the data acquisition process with the regulations. After optimizing the process model, full-scale data access takes place to exchange all data as contracted.

Each phase contains several activities that need to be agreed with each data owner individually. The particular activities and recommendations for their implementation were deduced from the results of the qualitative content analysis and are described in the following sections.

At the beginning of the project, the prerequisites to initiate and coordinate the data acquisition process are determined. In this paper, the terms “data owner” (DO) for the licensor of the data for secondary use, and “secondary user” (SU) for the licensee are used. This is in accordance with the recommendations of GPS [18]. Figure 1 [Fig. 1] depicts the sequentially performed tasks and the information exchange between the data owner and the secondary user. After the data owners confirm their willingness to collaborate with the secondary user, coordination on the specifications of the data that is to be provided follows.

Data characteristics (columns, profiles, and number of records) and data quality (individual level or aggregated) should be clearly specified in a DUA together with additional aspects concerning data usage, access, and storage. This includes the specification of underlying legal basis and individual policies of the data owner. Furthermore, all persons who will gain access to the data should be named and different roles and affiliated responsibilities should be defined. The DUA should also specify the location where data analyses take place. If the data are allowed to be transferred via the Internet, it should be stated how the data must be prepared (anonymized, pseudonymized, encrypted) and protected. In cases where legal regulations do not allow the transfer of data to the secondary user, it might be stated that the analyses can be conducted in the premises of the data owner. If data need to be archived further for retention reasons, it must be stipulated which party is responsible and how long the data will be archived. It should also be specified, whether the usage of the data is only allowed for a specific research project or if further analyses are allowed.

The draft of the DUA can be provided by the data owner or it can be developed by both parties. The secondary user has to check the conditions, in consultation with their legal advisers prior to signing the agreement.

After the prerequisites are regulated, the consented approach should be tested with a data sample (Figure 2 [Fig. 2]). The sample should be an excerpt of the full-scale data set and prepared in the same format (columns, profiles, variables, and value ranges). There are two different approaches in this phase:

1.

Data access is only provided in the premises of the data owner. In this case, the secondary user receives the excerpt to develop data analysis routines. A data processor accesses the database in the premises of the data owner later, executes the routines and transfers the results to the secondary user’s premises.

2.

Full-scale data sets are transferred to the secondary user for analyses. In this case, the excerpt is used to develop and test the data transfer approach. The transfer of the excerpt should be performed in the same way as that for the full-scale transfer, in order to detect problems or deviation from the regulations. In addition to the data sample, the data owner should provide a description of the data, including the description of the variables, attributes, and treatment codes of services.

As claims data are not originally collected for research purposes, their quality and validity, especially regarding completeness and plausibility, must be checked in all phases of the data acquisition process (Figure 3 [Fig. 3] and Figure 4 [Fig. 4]). The audit should be performed by the secondary user, to ensure the data are suitable for the planned analyses. After the data excerpt has been transferred to the secondary user, the data quality and validity can be assessed regarding technical and project-related aspects.

The technical audit involves the evaluation of completeness and plausibility of the parameters. Besides the content, the suitability of data transfer and storage approaches should be checked before full-scale data access and data analyses.

The project-related audit involves the evaluation of the suitability of the delivered profiles, columns, and records to perform the planned analyses.

Incorrect data should be identified and it should be checked if missing or incomplete data might be a problem. An upper limit of what is considered acceptable should be agreed. In case of inconsistencies, data owners should provide assistance and if needed, to deliver new corrected data.

The full-scale data sets contain all records that are consented with the data owners. It depends on whether full-scale analyses take place at the secondary user or at the premises of the data owner (Figure 4 [Fig. 4]). After scientific analyses have been performed, the plausibility of the results has to be evaluated during the analysis-specific audit.

The ADVOCATE project involved the acquisition of claims data from eight health insurance providers, health funds or health authorities from Denmark, Germany, Hungary, Ireland, the Netherlands, and the United Kingdom. To calculate the oral health care measures, the data owners were asked to provide data containing dental procedures, oral health outcomes, and socio-economic information. The data acquisition process was influenced by the fact that some of the data owners neither had experience in sharing their data for research nor had any predefined usage rules for them. Therefore, the four-phase process was developed to perform the data acquisition with each data owner in a comparable and reproducible way.

The data owners had already agreed to collaborate before the data acquisition process started. The necessary requirements and regulations regarding data protection and privacy were then determined with seven of the data owners and DUAs were concluded. Only one data owner did not require to determine a DUA. The agreements mostly applied country-specific legal regulations and internal policies of the data owners. Therefore, an individual agreement was negotiated with each data owner instead of consenting the same agreement with all data owners.

After DUAs were concluded, a qualitative content analysis was performed. The code structure covers characteristics concerning the provision of data by the data owner, data access and transfer, and scientific usage of the data. The entire code structure is depicted in Figure 5 [Fig. 5].

1.

Provision of data by the data owner: DUAs determined preparatory measures that the data owners used before data were shared for research. Either data were anonymized by removing specific attributes and aggregating values to make them indistinguishable or data were pseudonymized. DUAs also determined the content of the data.

2.

Data access and transfer: The specific regulations influenced data access and transfer. Seven data owners permitted data transfer to the secondary user via Internet, if data protection measures were implemented during transfer and storage at the receiving institute's premises. Only German regulations did not allow the transfer of data in general and Hungarian regulations did not allow the transfer of personal data.

3.

Scientific usage of the data by the secondary user: All DUAs only allowed using the data for the analyses in the ADVOCATE project and restricted further usage. Most of the data owners required that the data have to be returned or destroyed after the end of the ADVOCATE project or after a specific period. However, the data owners then committed to keep the original data sets for a specific period, in case the secondary user needs them again later. Inter-coder reliability was assessed, to check if both reviewers had the same understanding of the conditions mentioned in the documents. From the results of the coding, specific implementations were derived and developed afterwards.

An infrastructure for secure data handling and analysis was set up, using the collaborative data analysis platform AnalytiXagility [25], and individualized according to the requirements of the respective data owners. The analysis of the DUAs revealed that some policies of the data owners do not support the use of the collaborative data analysis platform. Furthermore, some data owners provided their own system or shared anonymized data via email.

Four of the data owners in the ADVOCATE project had initially shared a data sample. The data owners from Denmark, Germany, Ireland, and the Netherlands shared data excerpts from the full-scale data that contain between 100 and 500,000 cases. Other data owners shared an overview (descriptive statistics) of their data. Additionally, every data owner shared a description of the data, including the description of the variables, attributes, and treatment codes of dental services.

During the technical audit, completeness and types of columns, records, and profiles were evaluated. For example, one data set contained many records with edentulous patients, which appeared implausible. Hence, the data owners were contacted and asked for correction.

During the project-related audit, it was evaluated whether the planned analyses were possible with the provided data. In two cases, the data owners provided data that were too aggregated for the analyses. In these cases, the data owners were asked for less aggregated data, which was not possible due to privacy reasons.

Furthermore, in cooperation with dentists, it was evaluated whether the underlying treatment codes in the claims data were suitable to realize the planned analyses and whether they have the same meaning in the different reimbursement systems.

When the quality of the sample was adequate, the analyses of the full-scale data assets were performed (Figure 4 [Fig. 4]). Seven data owners sent the full-scale data sets to the secondary user. In the case of the German data owner, analysis scripts were developed using the data excerpt and executed on the full-scale data in the premises of the German data owner. Only aggregated results were exported. Some of the data sets contained millions of records from 2010 to date. Table 1 [Tab. 1] shows the characteristics of available data.

The use of claims data for research has great potential and can be used to evaluate effects of policy interventions on oral health outcomes [26]. However, accessing and using such data is challenging, because it often depends on individual coordination processes with the data owners [2]. Therefore, legal, ethical and technical prerequisites must be determined and agreed with each data owner separately. If multiple stakeholders are involved, researchers are confronted with additional challenges. Due to the lack of uniformity of legal regulations, it will rarely be possible to find a solution that fits all requirements of all data owners. A harmonized legal framework across and within Europe might facilitate the usage of claims data for research and increase data quality [27].

However, access to claims data will still be based on individual agreements with data owners. A heterogeneous data acquisition approach also poses a risk to data quality and the feasibility of analyses, because if data are provided on different quality levels, they might not be comparable or linkage with other data sources might not be possible [28], [29].

This paper has described how a generic approach has been developed, within the ADVOCATE project, to facilitate the data acquisition process and implement the individual requirements of each data owner.

There are comprehensive models for data acquisition and data quality assessment. For example, the controlled data sharing model of Prasser [30] describes a model, where secondary users can access data from a data owner via controlled data access environments. De Lusignan [31] describes six key concepts that researchers can use to assess the quality and utility of acquired routine data from data origin to curation and publication. Unlike these existing data acquisition and data quality assessment frameworks, the here presented approach was implemented pragmatically due to different regulations and different experience levels of the data owners in sharing data for research purposes. Whereas the Scottish health authorities provided a clearly defined application process for data usage [32], there were some data owners that neither had any experience in sharing their data for research nor had any predefined usage rules.

In this paper, it was shown that the data access in the ADVOCATE project was achieved by either accessing the data in the premises of the data owner or the data can be transferred to the secondary user. Besides these two approaches, there can be more complex ways to exchange data for secondary usage [33], [34].

By performing quality and validity audits and feedback of the results to the data owners, it was possible to affect the scope or quality of the delivered data in some cases. For example, some data owners delivered additional data when they were requested. In other cases, it was not possible to deliver different data, due to privacy reasons.

The experiences made during the ADVOCATE project show that a prescribed approach from the data owner is easier to implement, whereas the lack of experience on the side of the data owner gives an opportunity to tailor the data acquisition to the needs of the research project and may therefore become more flexible.

The process, described in this paper, can help researchers to overcome challenges with acquiring claims data from multiple data owners by providing a framework with general actions and processes. It also helps them to increase the efficiency and effectiveness of their processes. By defining the generic four-phase process in general and keeping the implementation open, it is possible to accommodate the individual requirements of all data owners.

The approach is limited insofar, as it only involves activities for data collection. It could be improved by extending it to include aspects of data harmonization to further increase data quality and to reduce risk of inability to perform data linkage. Also, it is solely based on experiences gained during the ADVOCATE project.

A standardized approach that supports researchers from the beginning of the process of data acquisition to data integration can help to achieve the highest possible improvement of knowledge through safe and efficient use of available data sources.

The ADVOCATE project has received funding from the European Commission’s Horizon 2020 research and innovation program under grant agreement 635183 (http://www.advocateoralhealth.com/).

We are grateful to Professor Helen Whelton, coordinator of the ADVOCATE Consortium, for her pivotal role in bringing people together and initiating the writing of the grant of the ADVOCATE project and for her comments and suggestions in the final proof-reading.

We thank the contributors to the ADVOCATE project:

• the ADVOCATE Scientific Advisory Board – Stephen Birch, Martin Chalkley, Roger Ellwood, Ekatarina Fabrikant, Jeffery Fellows, Christopher Fox, Frank Fox, Dympna Kavanagh, John Lavis, Roger Matthews, Mariano Sanz, Paula Vassalo, and Sandra White;
• the ADVOCATE General Assembly – Renske van der Kaden, Onno van der Galien, Lisa Bøge Christensen, Gail Douglas, Kenneth Eaton, Gerard Gavin, Geert van der Heijden, Jochen Walker, Stefan Listl, Gabor Nagy, Karen O’Hanlon, Andrew Taylor, Helen Whelton, Noel Woods;
• the ADVOCATE Ethics Advisory Board – Mary Donnelly, Eckart Feifel, Jon Fistein, Joakim Lilholt, Orsolya Peter, Elmar Reich, Evert-Ben van Veen, Agnes Zana;
• the ADVOCATE project coordinator Anita Blakeston; and
• the coworkers of the ADVOCATE project – Susan Ahern, Hanny Alwicher, Fatiha Baâdoudi, Robert Bryce, Julia Csikar, Frank Gabel, Melinda Madlena, Neal Maskrey, Emma McDonald, Kasper Rosing, Tanja Schamma, Anna-Lena Trescher, Karen Vinall-Collier, Elaine Ward, and Evangelos Zormpas.

The authors declare that they have no competing interests.