Sample size considerations and predictive performance of multinomial logistic prediction models

Valentijn M.T. de Jong*, Marinus J.C. Eijkemans, Ben van Calster, Dirk Timmerman, Karel G.M. Moons, Ewout W. Steyerberg, Maarten van Smeden

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

50 Citations (Scopus)


Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full-factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer.

Original languageEnglish
Pages (from-to)1601-1619
Number of pages19
JournalStatistics in Medicine
Issue number9
Early online date6 Jan 2019
Publication statusPublished - 1 Apr 2019


  • Multinomial logistic regression
  • overfit
  • prediction models
  • predictive performance
  • shrinkage
  • Multinomial Logistic Regression


Dive into the research topics of 'Sample size considerations and predictive performance of multinomial logistic prediction models'. Together they form a unique fingerprint.

Cite this