The Certainty Framework for Assessing Real-World Data in Studies of Medical Product Safety and Effectiveness

A fundamental question in using real-world data for clinical and regulatory decision making is: How certain must we be that the algorithm used to capture an exposure, outcome, cohort-defining characteristic, or confounder is what we intend it to be? We provide a practical framework to help researchers and regulators assess and classify the fit-for-purposefulness of real-world data by study variable for a range of data sources. The three levels of certainty (optimal, sufficient, and probable) must be considered in the context of each study variable, the specific question being studied, the study design, and the decision at hand.