gms | German Medical Science

67. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS), 13. Jahreskongress der Technologie- und Methodenplattform für die vernetzte medizinische Forschung e. V. (TMF)

21.08. - 25.08.2022, online

Central Statistical Monitoring approach and implementations on the German MS Registry

Meeting Abstract

  • Firas Fneish - Leibniz Universität Hannover, Hannover, Germany
  • David Ellenberger - MS-Register der DMSG, Bundeverband e.V., MS Forschungs- und Projektentwicklungs-gGmbH, Hannover, Germany
  • Niklas Frahm - MS-Register der DMSG, Bundeverband e.V., MS Forschungs- und Projektentwicklungs-gGmbH, Hannover, Germany
  • Alexander Stahmann - MS-Register der DMSG, Bundeverband e.V., MS Forschungs- und Projektentwicklungs-gGmbH, Hannover, Germany
  • Gerhard Fortwengel - Hochschule Hannover, Hannover, Germany
  • Frank Schaarschmidt - Leibniz Universität Hannover, Hannover, Germany

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 67. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS), 13. Jahreskongress der Technologie- und Methodenplattform für die vernetzte medizinische Forschung e.V. (TMF). sine loco [digital], 21.-25.08.2022. Düsseldorf: German Medical Science GMS Publishing House; 2022. DocAbstr. 82

doi: 10.3205/22gmds084, urn:nbn:de:0183-22gmds0849

Published: August 19, 2022

© 2022 Fneish et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at http://creativecommons.org/licenses/by/4.0/.


Outline

Text

Monitoring of clinical trials is a fundamental process required by regulatory agencies. It assures the compliance of a center to the required regulations and the trial protocol. Traditionally, monitoring teams relied on extensive on-site visits and source data verification. However, this is costly, and the outcome is limited. Thus, central statistical monitoring (CSM) is an additional approach recently embraced by ICH to detect problematic or erroneous data by using visualizations and statistical control measures. CSM serves to identify centers in need of additional (on-site) monitoring activities due to deviations. Researchers have been primarily focused on detecting inlier and outlier data. Other approaches include principal component analysis and distributions of the data. Here we focus on the utilization of comparisons of centers to the grand mean for different model types and assumptions for common data types, such as binomial, ordinal, and continuous response variables.

We applied methods that originate from the analysis of means (ANOM). Specifically, we implemented the usage of multiple comparisons of single centers to the grand mean of all centers. This approach is also available for various non-normal data types that are abundant in clinical trials. Further, using confidence intervals, an assessment of equivalence to the grand mean can be applied. For continuous outcomes, a linear model as well as a non-parametric approach are investigated. Generalized linear models (GLM), Bayesian linear models (BayesGLM), and bias-reduced generalized linear models (BrGLM) are applied for a binomial outcome. As for ordinal data, a non-parametric approach is assessed. In a Monte Carlo simulation study, the applied statistical approaches have been investigated for their ability to control type I error and the assessment of their respective power for balanced and unbalanced designs which are common in registry data and clinical trials. Data from the German MS-Register was used to verify the results on Real-World-Data (RWD).

Simulations covered a range of settings with number of centers varying between 5 and 10, while subjects per center vary between 2, 3, 4, 5, 6, 10, 20, 40, 50, 80, 100, 150 and 200. For each scenario a number of 1000 datasets were generated. Based on the simulation results, we give recommendations on the sample sizes needed for each scenario to control the type I error at a 5% level. With these requirements met, simulations showed that these methods control the type I error up to 5% and have sufficient power with increasing sample sizes. For small sample sizes however, the non-parametric method violates the type I error control and has also a decreased power for heavily unbalanced designs.

We were able to show how different statistical methods can be implemented to identify problematic centers in multi-center trials or registry data. The approach allows the recognition of centers that are significantly deviating from the average. Our approach serves the core purpose of CSM to improve the data integrity. Methods are illustrated on RWD from the German Multiple Sclerosis Registry including proportions of missing data, adverse events, disease severity scores and counts of relapses.

Competing interests:

  • Firas Fneish is an employee of Leibniz University Hannover and the German MS Registry.
  • David Ellenberger had no personal financial interests to disclose other than being employees of the German MS Registry.
  • Niklas Frahm is an employee of the MSFP. Moreover, he received travel funds for research meetings from Novartis.
  • Alexander Stahmann has no personal financial interests to disclose, other than being the leader of the German MS Registry, which receives (project) funding from a range of public and corporate sponsors, recently including G-BA, The German MS Trust, German MS Society, Biogen, Celgene (Bristol Myers Squibb), Merck, Novartis, Roche, Sanofi and Viatris.
  • Gerhard Fortwengel has nothing to disclose.
  • Frank Schaarschmidt has nothing to disclose.

The authors declare that an ethics committee vote is not required.


References

1.
EMA E6 (R2) Good Clinical Practice: Integrated Addendum to ICH E6 (R1) - Guidance for Industry - Guideline for Good Clinical Practice - E6 (R2). 2018.
2.
Agresti A. Categorical Data Analysis. Third Edition. Hoboken, New Jersey: John Wiley and Sons; 2013.
3.
Konietschke F, Hothorn LA, Brunner E. Rank-based multiple test procedures and simultaneous confidence intervals. Electronic Journal of Statistics. 2012;6:738-759. DOI: 10.1214/12-EJS691 External link
4.
Kosmidis I. brglm: Bias Reduction in Binary-Response Generalized Linear Models. R package version 0.7.2. 2021. Available from: https://cran.r-project.org/package=brglm External link
5.
Kosmidis I, Firth D. Bias reduction in exponential family nonlinear models. Biometrika. 2009;96(4):793-804.
6.
Kosmidis I, Firth D. Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models. Biometrika. 2021;108(1):71-82.
7.
Ohle LM, Ellenberger D, Flachenecker P, Friede T, Haas J, Hellwig K, Parciak T, Warnke C, Paul F, Zettl UK, Stahmann A. Chances and challenges of a long-term data repository in multiple sclerosis: 20th birthday of the German MS registry. Sci Rep. 2021;11:13340. DOI: 10.1038/s41598-021-92722-x External link
8.
R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021. Available from: https://www.R-project.org/ External link