Assessing Overall Survival Benefits in Advanced Cancers: The Role of External Comparator Cohort Studies with Real-World Data

External comparator cohort (ECC) studies with real-world data (RWD) may provide more reliable estimates of treatment differences compared to single-arm trials (SAT), yet they face limitations such as selection bias and data heterogeneity. This study assessed the perceived strength of evidence of ECC studies compared to SAT and randomized controlled studies (RCT). The study included healthcare professionals (HCP) from the European Hematology Association (EHA), the European Society for Medical Oncology (ESMO), and assessors from international regulatory agencies (RA). A conjoint analysis evaluated strength of evidence ratings for establishing an effect on OS for different hypothetical scenarios, based on different designs, RWD quality, and observed OS improvement, for a new cancer treatment for advanced disease and no effective treatments. Participants from HCP organizations rated RWD studies favorably (advantages outweigh disadvantages) more frequently (47.6%; n = 103) compared to RA participants (12.9%; n = 116). Compared to a SAT, a high-quality RWD ECC study showing a 1.5-month and 3-month OS improvement had 2.7 (95% CI: 1.9–3.8) and 14.7 (95% CI: 10.0–21.5) times higher odds of receiving a higher strength of evidence rating, respectively. The OR for RCT v. SAT was 36.4 (95% CI: 24.0–55.2) and 358.4 (95% CI: 217.3–591.3), respectively. Strength of evidence ratings were associated with maximum acceptable risk of severe or symptomatic toxicity. In conclusion, when evaluating the OS of new therapies, ECC studies with RWD, especially when based on high-quality RWD or demonstrating a larger OS benefit, were rated as more convincing than SAT without a formal control.