TY - JOUR
T1 - Evaluation of data imputation strategies in complex, deeply-phenotyped data sets
T2 - the case of the EU-AIMS Longitudinal European Autism Project
AU - Llera, A
AU - Brammer, M
AU - Oakley, B
AU - Tillmann, J
AU - Zabihi, M
AU - Amelink, J S
AU - Mei, T
AU - Charman, T
AU - Ecker, C
AU - Dell'Acqua, F
AU - Banaschewski, T
AU - Moessnang, C
AU - Baron-Cohen, S
AU - Holt, R
AU - Durston, S
AU - Murphy, D
AU - Loth, E
AU - Buitelaar, J K
AU - Floris, D L
AU - Beckmann, C F
N1 - Funding Information:
This project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115300 (for EU-AIMS) and No 777394 (for AIMS-2-TRIALS). This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA and AUTISM SPEAKS, Autistica, SFARI. This work has also been supported by the Horizon2020 programme CANDY Grant No. 847818). DLF is supported by funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 101025785. This work was also supported by the Netherlands Organization for Scientific Research through VICI grant (Grant No. 17854 [to CFB]). The research leading to the presented work has received funding from the developing Human Connectome Project (dHCP) through a Synergy Grant by the European Research Council under the European Union’s Seventh Framework Programme (FP/2007–2013), ERC Grant Agreement no. 319456. We also gratefully acknowledge funding from the Wellcome Collaborative Award (215573/Z/19/Z).
Publisher Copyright:
© 2022, The Author(s).
PY - 2022/8/16
Y1 - 2022/8/16
N2 - An increasing number of large-scale multi-modal research initiatives has been conducted in the typically developing population, e.g. Dev. Cogn. Neur. 32:43-54, 2018; PLoS Med. 12(3):e1001779, 2015; Elam and Van Essen, Enc. Comp. Neur., 2013, as well as in psychiatric cohorts, e.g. Trans. Psych. 10(1):100, 2020; Mol. Psych. 19:659-667, 2014; Mol. Aut. 8:24, 2017; Eur. Child and Adol. Psych. 24(3):265-281, 2015. Missing data is a common problem in such datasets due to the difficulty of assessing multiple measures on a large number of participants. The consequences of missing data accumulate when researchers aim to integrate relationships across multiple measures. Here we aim to evaluate different imputation strategies to fill in missing values in clinical data from a large (total N = 764) and deeply phenotyped (i.e. range of clinical and cognitive instruments administered) sample of N = 453 autistic individuals and N = 311 control individuals recruited as part of the EU-AIMS Longitudinal European Autism Project (LEAP) consortium. In particular, we consider a total of 160 clinical measures divided in 15 overlapping subsets of participants. We use two simple but common univariate strategies-mean and median imputation-as well as a Round Robin regression approach involving four independent multivariate regression models including Bayesian Ridge regression, as well as several non-linear models: Decision Trees (Extra Trees., and Nearest Neighbours regression. We evaluate the models using the traditional mean square error towards removed available data, and also consider the Kullback-Leibler divergence between the observed and the imputed distributions. We show that all of the multivariate approaches tested provide a substantial improvement compared to typical univariate approaches. Further, our analyses reveal that across all 15 data-subsets tested, an Extra Trees regression approach provided the best global results. This not only allows the selection of a unique model to impute missing data for the LEAP project and delivers a fixed set of imputed clinical data to be used by researchers working with the LEAP dataset in the future, but provides more general guidelines for data imputation in large scale epidemiological studies.
AB - An increasing number of large-scale multi-modal research initiatives has been conducted in the typically developing population, e.g. Dev. Cogn. Neur. 32:43-54, 2018; PLoS Med. 12(3):e1001779, 2015; Elam and Van Essen, Enc. Comp. Neur., 2013, as well as in psychiatric cohorts, e.g. Trans. Psych. 10(1):100, 2020; Mol. Psych. 19:659-667, 2014; Mol. Aut. 8:24, 2017; Eur. Child and Adol. Psych. 24(3):265-281, 2015. Missing data is a common problem in such datasets due to the difficulty of assessing multiple measures on a large number of participants. The consequences of missing data accumulate when researchers aim to integrate relationships across multiple measures. Here we aim to evaluate different imputation strategies to fill in missing values in clinical data from a large (total N = 764) and deeply phenotyped (i.e. range of clinical and cognitive instruments administered) sample of N = 453 autistic individuals and N = 311 control individuals recruited as part of the EU-AIMS Longitudinal European Autism Project (LEAP) consortium. In particular, we consider a total of 160 clinical measures divided in 15 overlapping subsets of participants. We use two simple but common univariate strategies-mean and median imputation-as well as a Round Robin regression approach involving four independent multivariate regression models including Bayesian Ridge regression, as well as several non-linear models: Decision Trees (Extra Trees., and Nearest Neighbours regression. We evaluate the models using the traditional mean square error towards removed available data, and also consider the Kullback-Leibler divergence between the observed and the imputed distributions. We show that all of the multivariate approaches tested provide a substantial improvement compared to typical univariate approaches. Further, our analyses reveal that across all 15 data-subsets tested, an Extra Trees regression approach provided the best global results. This not only allows the selection of a unique model to impute missing data for the LEAP project and delivers a fixed set of imputed clinical data to be used by researchers working with the LEAP dataset in the future, but provides more general guidelines for data imputation in large scale epidemiological studies.
KW - Autistic Disorder/genetics
KW - Bayes Theorem
KW - Child
KW - Data Collection/methods
KW - Humans
KW - Clinical data
KW - Machine learning
KW - Imputation
KW - Multivariate
UR - http://www.scopus.com/inward/record.url?scp=85136074203&partnerID=8YFLogxK
U2 - 10.1186/s12874-022-01656-z
DO - 10.1186/s12874-022-01656-z
M3 - Article
C2 - 35971088
SN - 1471-2288
VL - 22
JO - BMC Medical Research Methodology
JF - BMC Medical Research Methodology
IS - 1
M1 - 229
ER -