Outlier classification performance of risk adjustment methods when profiling multiple providers

Timo B. Brakenhoff*, Kit C.B. Roes, Karel G.M. Moons, Rolf H.H. Groenwold

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

8 Downloads (Pure)


Background: When profiling multiple health care providers, adjustment for case-mix is essential to accurately classify the quality of providers. Unfortunately, misclassification of provider performance is not uncommon and can have grave implications. Propensity score (PS) methods have been proposed as viable alternatives to conventional multivariable regression. The objective was to assess the outlier classification performance of risk adjustment methods when profiling multiple providers. Methods: In a simulation study based on empirical data, the classification performance of logistic regression (fixed and random effects), PS adjustment, and three PS weighting methods was evaluated when varying parameters such as the number of providers, the average incidence of the outcome, and the percentage of outliers. Traditional classification accuracy measures were considered, including sensitivity and specificity. Results: Fixed effects logistic regression consistently had the highest sensitivity and negative predictive value, yet a low specificity and positive predictive value. Of the random effects methods, PS adjustment and random effects logistic regression performed equally well or better than all the remaining PS methods for all classification accuracy measures across the studied scenarios. Conclusions: Of the evaluated PS methods, only PS adjustment can be considered a viable alternative to random effects logistic regression when profiling multiple providers in different scenarios.

Original languageEnglish
Article number54
JournalBMC Medical Research Methodology
Issue number1
Publication statusPublished - 15 Jun 2018


  • Classification
  • Logistic regression
  • Profiling
  • Propensity score
  • Random effects
  • Risk adjustment
  • Simulation study


Dive into the research topics of 'Outlier classification performance of risk adjustment methods when profiling multiple providers'. Together they form a unique fingerprint.

Cite this