Quality control, data cleaning, imputation

Dawei Liu, Hanne I. Oberman, Johanna Muñoz, Jeroen Hoogland, Thomas P.A. Debray*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

Abstract

This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.

Original languageEnglish
Title of host publicationClinical Applications of Artificial Intelligence in Real-World Data
EditorsFolkert W. Asselbergs, Spiros Denaxas, Daniel L. Oberski, Jason H. Moore
PublisherSpringer
Pages7-36
Number of pages30
Edition1
ISBN (Electronic)9783031366789
ISBN (Print)9783031366772
DOIs
Publication statusPublished - 5 Nov 2023

Keywords

  • Conditional modelling imputation
  • Heckman selection model
  • Imputation
  • Informative missingness
  • Joint modelling imputation
  • Machine learning imputation
  • Matrix completion
  • Missing at random
  • Missing completely at random
  • Missing data
  • Missing indicator
  • Missing not at random
  • Nearest neighbor
  • Neural networks
  • Pattern submodels
  • Rubin's rules
  • Sporadically missing
  • Support vector machines
  • Surrogate splits
  • Systematically missing
  • Tree-based ensembles

Cite this