gms | German Medical Science

Background: A positive deviance approach requires the identification of positive outlier organizations within an organizational field. It presumes that barriers to adoption of innovations can be overcome by learning from positive outliers who are able to overcome barriers and ultimately adopt the new practice in spite of having similar adoption constraints. However, there is currently no gold standard for identifying “positive deviants” (PD) in health services research.

Research Objective: The aim of this study was to examine an empirically-driven approach to identify PDs. In particular, our objective was to identify physician organizations participating in Accountable Care Organizations (ACOs) with robust performance management systems (PMS) compared to similarly structured practices. We determined PDs for the purpose of conducting subsequent case studies to examine how practices innovate and implement their PMS and to understand the role of their ACO as a facilitator of PMS development.

Methods: The third National Survey of Physician Organizations (NSPO3), a nationally representative sample of physician practices (n=1,398 for a response rate of 49.7%) was analyzed for this study. Data on practice characteristics, care management processes, and other variables were collected via a 40-minute phone survey from January 2012–Nov 2013. We excluded organizations not participating in an ACO and those with missing values, resulting in an analytic sample of 316 organizations. We then constructed a 20-item composite PMS index using principal factor analysis. Linear regression estimated in a first model (M₁) the impact of contextual characteristics (over which practices have little or no control in the short term), such as external incentives or practice size, on increases in practices’ PMS index. In a second model (M₂) we added internal mechanisms (processes where practices have more control over the implementation), such as health information technology (HIT) functionality or the participation in quality improvement (QI) learning collaboratives. Next, practices were sorted based on their residual value (distance from the predicted value). Then, we identified PD as organizations with the largest residuals. We used the 90th percentile of the residuals as cut-off point for inclusion as PD. In addition, practices had to be in the 90th percentile of the PMS index and their affiliated ACO had capabilities for performance measurement. All analyses are weighted.

Results: Including internal mechanism variables in the second regression model improved model goodness of fit (M₁ R²=0.65, M₂ R²=0.80, adjusted Wald test: p=0.00), indicating that high HIT functionality, chronic disease registries and the participation in QI learning collaboratives contribute to development of robust PMS systems. Residuals in the second model ranged from -0.54 to 0.59; 91 PD practices (= 28.8% of the sample) were identified by using the 90th percentile of the residuals as cut-off point for defining positive deviance. After excluding practices that did not meet the 90th percentile of the PMS index (n=50), 41 practices were retained (13.0%). After excluding organizations where their affiliated ACO did not have the capabilities for performance measurement (n=21), a final set of 20 (6.3%) cases were ultimately classified as PDs.

Discussion: There is no consistent definition of positive deviance or approach to identify PDs in organizational research. Identifying PDs often relies on self-report or is sometimes conflated with high performance. We used an empirically based positive deviance approach to determine practices participating in ACOs with PMS that are more developed than predicted based on the practice’s contextual characteristics and internal mechanisms. The unexplained deviance from the predicted value of the PDs suggests that there may be underlying best-practices in these organizations that we did not measure. We are currently recruiting PD organizations to gain in-depth knowledge about these unmeasured factors that enable the development and maintenance of PMS in spite of facing constraints faced by practices unable to develop robust PMS. Preliminary findings from these case studies will be highlighted.

Implications for Policy or Practice: We developed an empirically-driven approach to identify PD practices that were able to overcome constraints for deeper assessment. Organizations that develop robust PMS despite facing barriers may generate more innovative and generalizable insights for peers that encounter similar restricting characteristics and can provide insight into broader dissemination of evidence-based organizational structures and processes across organizational fields. This empirical method to identify PDs and insights from the US ACO experience can also be extended to physician networks and population-based integrated care models in other countries experimenting with these organization forms, such as Germany.