gms | German Medical Science

Background: The standardized mortality ratio (SMR) is often used to assess and compare hospital performance. While it has been recognized that hospitals may differ in their SMRs due to differences in patient composition, there is a lack of rigorous analysis of this and other – largely unrecognized – properties of the SMR.

Objectives:

1.

To propose a set of axiomatic requirements that should be fulfilled by well-behaved standardized mortality measures.

2.

To analyze and evaluate basic properties of the SMR in the light of the proposed axiomatic requirements.

Methods: Effects of variations in patient composition, hospital size, and actual and expected mortality rates on the SMR were examined using basic algebra and calculus. In this regard, we distinguished between standardization using expected mortality rates derived from a different dataset (external standardization) and standardization based on a dataset including the considered hospitals (internal standardization). The results were illustrated by hypothetical examples.

Results: Based on a set of proposed desirable properties of standardized mortality measures, five axiomatic requirements for those measures were derived (strict monotonicity, case-mix insensitivity, scale insensitivity, equivalence principle, dominance principle). Formal analysis of the SMR showed that hospitals with different SMRs are differently affected by the same variations in case mix under external standardization. While the SMR is insensitive to hospital size under external standardization, the hospital’s case mix influences effects of variations in both actual and expected mortality rates. Effects of variations in the latter additionally depend on the size of the SMR. Under internal standardization, the SMR is sensitive to both case-mix and hospital size. Paradoxically, a higher stratum-specific mortality rate may induce a lower SMR under internal standardization if the hospital’s SMR exceeds 1 and the hospital accounts for a large share of the stratum’s total number of patients. In summary, the SMR fulfills only two (none) out of the five proposed axiomatic requirements under external (internal) standardization.

Discussion: The SMR does not fulfill most of the proposed axiomatic requirements under both standardization approaches. Generally, the SMRs of hospitals are differently affected by variations in case mix and actual and expected mortality rates unless the hospitals are identical in these characteristics. These properties hamper valid assessment and comparison of hospital performance based on the SMR.

Practical implications: In the presence of large heterogeneity of the analyzed hospitals in terms of case mix and mortality rates, the SMR cannot be trusted. As a general recommendation, empirical studies therefore should assess and report the degree of heterogeneity of the considered hospitals and take effects of heterogeneity into account when interpreting calculated SMRs.