gms | German Medical Science

Personal data is often scattered across various stakeholders due to its collection for various data collection purposes. This leads to a high degree of fragmentation, which necessitates the consolidation of multiple data sources in order to obtain a complete view of natural persons [1]. Linking personal data records together is trivial with a globally unique personal identifier, but such an identifier is often either not available or out of scope in most scenarios. Algorithms from the field of record linkage have therefore been employed instead. They operate on identification data and assign a similarity to record pairs in order to decide whether they should be merged or not.

These record linkage algorithms require testing on realistic data to evaluate their efficacy in real-world situations [2]. However due to the sensitive nature of identification data, access to real-world testing data has been mostly exclusive to researchers with personal ties to medical institutions in the past [3], [4]. This has led to the creation of tools which generate personal data that seems realistic based on publicly available data sources. To the best of our knowledge, all previously published tools are either inactive, unmaintained, closed source or outdated.

We present Gecko: an open-source Python library for the generation and mutation of personal identification data based on public data and error sources. It takes after GeCo which showed the promise of creating reproducible and shareable scripts to generate data [5]. The ease of integration into data science applications of the original library leaves a lot to be desired. Gecko addresses this by reimplementing GeCo’s core features on top of popular data science libraries and extending them by fixing GeCo’s limitations, allowing the generation of arbitrarily complex multivariate data, fine-grain control over its randomized routines and data mutation across multiple instead of single fields.

Gecko makes extensive use of Pandas data frames which allow exports of generated data in various interoperable file formats such as CSV. We validated that data generated by Gecko can be imported into E-PIX, which ensures Gecko’s compatibility with other tools with CSV parsing capabilities. Furthermore, we extensively benchmarked Gecko to ensure that it fulfills its performance claims. Despite the lack of test data from other solutions in the field, we estimate that Gecko’s single-core performance stands as best-in-class by a comfortable margin.

Gecko’s performance and configurability allows it to generate datasets with millions of records for the validation of record linkage algorithms in reasonable time frames. Its capabilities to quickly generate data on-the-fly opens it up for use in other data science applications where realistic identification data may be needed. A publicly available data repository allows for quick testing of Gecko’s capabilities for library users. We encourage users of Gecko to donate data samples from various regions and languages in order to obtain higher multilingual coverage. Future versions of Gecko aim at providing export facilities for FHIR, as well as support for more complex error classes such as temporal errors and column shifts in data.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.