Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases

Background: Dysplasia in Barrett's esophagus (BE) biopsies is associated with low observer agreement among general pathologists. Therefore, expert review is advised. We are developing a web-based, national expert review panel for histological review of BE biopsies. Objective: The aim of this study was to create benchmark quality criteria for future members. Methods: Five expert BE pathologists, with 10–30 years of BE experience, weekly handling 5–10 cases (25% dysplastic), assessed a case set of 60 digitalized cases, enriched for dysplasia. Each case contained all slides from one endoscopy (non-dysplastic BE (NDBE), n = 21; low-grade dysplasia (LGD), n = 20; high-grade dysplasia (HGD), n = 19). All cases were randomized and assessed twice followed by group discussions to create a consensus diagnosis. Outcome measures: percentage of ‘indefinite for dysplasia’ (IND) diagnoses, intra-observer agreement, and agreement with the consensus ‘gold standard’ diagnosis. Results: Mean percentage of IND diagnoses was 8% (3–14%) and mean intra-observer agreement was 0.84 (0.66–1.02). Mean agreement with the consensus diagnosis was 90% (95% prediction interval (PI) 82–98%). Conclusion: Expert pathology review of BE requires the scoring of a limited number of IND cases, consistency of assessment and a high agreement with a consensus gold standard diagnosis. These benchmark quality criteria will be used to assess the performance of other pathologists joining our panel.