gms | German Medical Science

Background and Purpose: Provider profiling (PP) is a measure of provider performance (e.g. hospital) and used as a steering mechanism to improve quality of care. Methodological considerations need to be taken into account to receive valid and reliable profiling results. The aim of this study is to describe the available methodology and apply three methods of PP to empirical data.

Research Focus: Patient survey data (n=1661) assessing experiences with hospital stays from a cross-sectional study with 123 units from 23 Swiss hospitals were used. We applied intraclass correlation 1 (ICC1, describing variability between providers) and intraclass correlation 2 (ICC2, assessing reliability of the measure through measurement errors) to explore variation between providers and reliability of the profiling. In consideration of risk adjustments due to different patient and hospital characteristics, we apply three different PP techniques: observed-over-expected, predicted-over-expected and empirical bias estimators.

Results: For patient experience variables, the ICC1 on hospital level ranged from 0.003 to 0.029 and on unit level from 0 to 0.041, indicating no to small variability. ICC2 on the hospital level ranged from 0.40 to 0.50 and on unit level from 0.20 to 0.96 indicating fair to moderate reliability. Depending on the applied PP methods hospitals and units were ranked differently.

Conclusions: With our example we critically discuss the utilisation of patient experience variables for PP. Due to limited between-provider variation and reliability, the value of patient experience measures for PP is questionable. Variability and reliability are prerequisites to compare hospitals and receive credible PP results.