Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases

M. J. van der Wel; L. C. Duits; E. Klaver; R. E. Pouw; C. A. Seldenrijk; G. J.A. Offerhaus; M. Visser; F. J.W. ten Kate; J. G. Tijssen; J. J.G.H.M. Bergman; S. L. Meijer

doi:10.1177/2050640618764710

Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases

M. J. van der Wel^*, L. C. Duits, E. Klaver, R. E. Pouw, C. A. Seldenrijk, G. J.A. Offerhaus, M. Visser, F. J.W. ten Kate, J. G. Tijssen, J. J.G.H.M. Bergman, S. L. Meijer

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

1 Downloads (Pure)

Abstract

Background: Dysplasia in Barrett's esophagus (BE) biopsies is associated with low observer agreement among general pathologists. Therefore, expert review is advised. We are developing a web-based, national expert review panel for histological review of BE biopsies. Objective: The aim of this study was to create benchmark quality criteria for future members. Methods: Five expert BE pathologists, with 10–30 years of BE experience, weekly handling 5–10 cases (25% dysplastic), assessed a case set of 60 digitalized cases, enriched for dysplasia. Each case contained all slides from one endoscopy (non-dysplastic BE (NDBE), n = 21; low-grade dysplasia (LGD), n = 20; high-grade dysplasia (HGD), n = 19). All cases were randomized and assessed twice followed by group discussions to create a consensus diagnosis. Outcome measures: percentage of ‘indefinite for dysplasia’ (IND) diagnoses, intra-observer agreement, and agreement with the consensus ‘gold standard’ diagnosis. Results: Mean percentage of IND diagnoses was 8% (3–14%) and mean intra-observer agreement was 0.84 (0.66–1.02). Mean agreement with the consensus diagnosis was 90% (95% prediction interval (PI) 82–98%). Conclusion: Expert pathology review of BE requires the scoring of a limited number of IND cases, consistency of assessment and a high agreement with a consensus gold standard diagnosis. These benchmark quality criteria will be used to assess the performance of other pathologists joining our panel.

Original language	English
Pages (from-to)	830-837
Number of pages	8
Journal	United European Gastroenterology Journal
Volume	6
Issue number	6
DOIs	https://doi.org/10.1177/2050640618764710
Publication status	Published - Jul 2018

Keywords

Barrett's esophagus
benchmark quality criteria
digital microscopy
intra-observer agreement
low-grade dysplasia
review panel
whole slide imaging

Access to Document

10.1177/2050640618764710

DevelopmentFinal published version, 157 KB

Cite this

van der Wel, M. J., Duits, L. C., Klaver, E., Pouw, R. E., Seldenrijk, C. A., Offerhaus, G. J. A., Visser, M., ten Kate, F. J. W., Tijssen, J. G., Bergman, J. J. G. H. M., & Meijer, S. L. (2018). Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases. United European Gastroenterology Journal, 6(6), 830-837. https://doi.org/10.1177/2050640618764710

@article{7ade61281c5d47fe9069022132ad78b0,

title = "Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases",

abstract = "Background: Dysplasia in Barrett's esophagus (BE) biopsies is associated with low observer agreement among general pathologists. Therefore, expert review is advised. We are developing a web-based, national expert review panel for histological review of BE biopsies. Objective: The aim of this study was to create benchmark quality criteria for future members. Methods: Five expert BE pathologists, with 10–30 years of BE experience, weekly handling 5–10 cases (25% dysplastic), assessed a case set of 60 digitalized cases, enriched for dysplasia. Each case contained all slides from one endoscopy (non-dysplastic BE (NDBE), n = 21; low-grade dysplasia (LGD), n = 20; high-grade dysplasia (HGD), n = 19). All cases were randomized and assessed twice followed by group discussions to create a consensus diagnosis. Outcome measures: percentage of {\textquoteleft}indefinite for dysplasia{\textquoteright} (IND) diagnoses, intra-observer agreement, and agreement with the consensus {\textquoteleft}gold standard{\textquoteright} diagnosis. Results: Mean percentage of IND diagnoses was 8% (3–14%) and mean intra-observer agreement was 0.84 (0.66–1.02). Mean agreement with the consensus diagnosis was 90% (95% prediction interval (PI) 82–98%). Conclusion: Expert pathology review of BE requires the scoring of a limited number of IND cases, consistency of assessment and a high agreement with a consensus gold standard diagnosis. These benchmark quality criteria will be used to assess the performance of other pathologists joining our panel.",

keywords = "Barrett's esophagus, benchmark quality criteria, digital microscopy, intra-observer agreement, low-grade dysplasia, review panel, whole slide imaging",

author = "{van der Wel}, {M. J.} and Duits, {L. C.} and E. Klaver and Pouw, {R. E.} and Seldenrijk, {C. A.} and Offerhaus, {G. J.A.} and M. Visser and {ten Kate}, {F. J.W.} and Tijssen, {J. G.} and Bergman, {J. J.G.H.M.} and Meijer, {S. L.}",

year = "2018",

month = jul,

doi = "10.1177/2050640618764710",

language = "English",

volume = "6",

pages = "830--837",

journal = "United European Gastroenterology Journal",

issn = "2050-6406",

publisher = "John Wiley & Sons Inc.",

number = "6",

}

van der Wel, MJ, Duits, LC, Klaver, E, Pouw, RE, Seldenrijk, CA, Offerhaus, GJA, Visser, M, ten Kate, FJW, Tijssen, JG, Bergman, JJGHM & Meijer, SL 2018, 'Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases', United European Gastroenterology Journal, vol. 6, no. 6, pp. 830-837. https://doi.org/10.1177/2050640618764710

TY - JOUR

T1 - Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases

AU - van der Wel, M. J.

AU - Duits, L. C.

AU - Klaver, E.

AU - Pouw, R. E.

AU - Seldenrijk, C. A.

AU - Offerhaus, G. J.A.

AU - Visser, M.

AU - ten Kate, F. J.W.

AU - Tijssen, J. G.

AU - Bergman, J. J.G.H.M.

AU - Meijer, S. L.

PY - 2018/7

Y1 - 2018/7

N2 - Background: Dysplasia in Barrett's esophagus (BE) biopsies is associated with low observer agreement among general pathologists. Therefore, expert review is advised. We are developing a web-based, national expert review panel for histological review of BE biopsies. Objective: The aim of this study was to create benchmark quality criteria for future members. Methods: Five expert BE pathologists, with 10–30 years of BE experience, weekly handling 5–10 cases (25% dysplastic), assessed a case set of 60 digitalized cases, enriched for dysplasia. Each case contained all slides from one endoscopy (non-dysplastic BE (NDBE), n = 21; low-grade dysplasia (LGD), n = 20; high-grade dysplasia (HGD), n = 19). All cases were randomized and assessed twice followed by group discussions to create a consensus diagnosis. Outcome measures: percentage of ‘indefinite for dysplasia’ (IND) diagnoses, intra-observer agreement, and agreement with the consensus ‘gold standard’ diagnosis. Results: Mean percentage of IND diagnoses was 8% (3–14%) and mean intra-observer agreement was 0.84 (0.66–1.02). Mean agreement with the consensus diagnosis was 90% (95% prediction interval (PI) 82–98%). Conclusion: Expert pathology review of BE requires the scoring of a limited number of IND cases, consistency of assessment and a high agreement with a consensus gold standard diagnosis. These benchmark quality criteria will be used to assess the performance of other pathologists joining our panel.

AB - Background: Dysplasia in Barrett's esophagus (BE) biopsies is associated with low observer agreement among general pathologists. Therefore, expert review is advised. We are developing a web-based, national expert review panel for histological review of BE biopsies. Objective: The aim of this study was to create benchmark quality criteria for future members. Methods: Five expert BE pathologists, with 10–30 years of BE experience, weekly handling 5–10 cases (25% dysplastic), assessed a case set of 60 digitalized cases, enriched for dysplasia. Each case contained all slides from one endoscopy (non-dysplastic BE (NDBE), n = 21; low-grade dysplasia (LGD), n = 20; high-grade dysplasia (HGD), n = 19). All cases were randomized and assessed twice followed by group discussions to create a consensus diagnosis. Outcome measures: percentage of ‘indefinite for dysplasia’ (IND) diagnoses, intra-observer agreement, and agreement with the consensus ‘gold standard’ diagnosis. Results: Mean percentage of IND diagnoses was 8% (3–14%) and mean intra-observer agreement was 0.84 (0.66–1.02). Mean agreement with the consensus diagnosis was 90% (95% prediction interval (PI) 82–98%). Conclusion: Expert pathology review of BE requires the scoring of a limited number of IND cases, consistency of assessment and a high agreement with a consensus gold standard diagnosis. These benchmark quality criteria will be used to assess the performance of other pathologists joining our panel.

KW - Barrett's esophagus

KW - benchmark quality criteria

KW - digital microscopy

KW - intra-observer agreement

KW - low-grade dysplasia

KW - review panel

KW - whole slide imaging

UR - http://www.scopus.com/inward/record.url?scp=85043330350&partnerID=8YFLogxK

U2 - 10.1177/2050640618764710

DO - 10.1177/2050640618764710

M3 - Article

C2 - 30023060

AN - SCOPUS:85043330350

SN - 2050-6406

VL - 6

SP - 830

EP - 837

JO - United European Gastroenterology Journal

JF - United European Gastroenterology Journal

IS - 6

ER -

Development of benchmark quality criteria for assessing whole-endoscopy Barrett's esophagus biopsy cases

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this