TY - JOUR
T1 - External validation of WGS-based antimicrobial susceptibility prediction tools, KOVER-AMR and ResFinder 4.1, for Escherichia coli clinical isolates
AU - Verschuuren, Tess
AU - Bosch, Thijs
AU - Mascaro, Valentina
AU - Willems, Rob
AU - Kluytmans, Jan
N1 - Funding Information:
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. All authors have nothing to declare. All generated raw reads were submitted to the European Nucleotide Archive (ENA) of the European Bioinformatics Institute (EBI) under the study accession number PRJEB35000. All phenotypes are available in the supplementary material ( Supplementary material 2 ). This study was internally funded by the Netherlands National Institute for Public Health and the Environment ( RIVM ) and the University Medical Center Utrecht (UMCU) and the authors received no specific funding for this work.
Funding Information:
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. All authors have nothing to declare. All generated raw reads were submitted to the European Nucleotide Archive (ENA) of the European Bioinformatics Institute (EBI) under the study accession number PRJEB35000. All phenotypes are available in the supplementary material (Supplementary material 2). This study was internally funded by the Netherlands National Institute for Public Health and the Environment (RIVM) and the University Medical Center Utrecht (UMCU) and the authors received no specific funding for this work.
Publisher Copyright:
© 2022 The Authors
PY - 2022/11
Y1 - 2022/11
N2 - Objective: To externally validate whole genome sequence-antimicrobial susceptibility testing phenotype prediction tools KOVER-AMR and ResFinder 4.1 for Escherichia coli clinical isolates from Dutch routine care. Methods: A random sample of 234 E. coli and 283 third generation cephalosporin-resistant E. coli isolates from urine and blood were collected (2014–2017). Culture-antimicrobial susceptibility testing was performed using VITEK 2 and BD Phoenix. Sequences were used as input for KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1. The concordance, major error rate (MER), and very major error rate (VMER) were calculated, with subsequent comparison to U.S. Food and Drug Administration (FDA) criteria (MER ≤3% and VMER with a 95% confidence interval ≤1.5–≤7.5%). Results: ResFinder 4.1 performed better than KOVER-AMR-models; however, neither tool achieved overall (V)MERs below FDA criteria. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1, MER (cumulative all antimicrobials) were: 5.1% (4.4–5.9), 4.3% (3.6–5.0), and 5.1% (4.5–5.8), respectively. MERs ≤3% were achieved for 6 (SCM) and 5 (CART) of the 11 tested antimicrobials for KOVER-AMR-models and for 9/13 antimicrobials tested with ResFinder 4.1. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1 cumulative VMERs were: 26% (24–28), 29% (27–31), and 11% (9.2–12). VMERs with a 95% CI ≤ 1.5–≤7.5 were only achieved for 4/13 tested antimicrobials with ResFinder 4.1. Discussion: In this study, whole genome sequence-antimicrobial susceptibility testing phenotype prediction tools KOVER-AMR and ResFinder 4.1 did not meet the FDA criteria needed for clinical diagnostic use in 517 E. coli clinical isolates from Dutch routine care. The tested tools should be further improved before they can be used for clinical decision making.
AB - Objective: To externally validate whole genome sequence-antimicrobial susceptibility testing phenotype prediction tools KOVER-AMR and ResFinder 4.1 for Escherichia coli clinical isolates from Dutch routine care. Methods: A random sample of 234 E. coli and 283 third generation cephalosporin-resistant E. coli isolates from urine and blood were collected (2014–2017). Culture-antimicrobial susceptibility testing was performed using VITEK 2 and BD Phoenix. Sequences were used as input for KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1. The concordance, major error rate (MER), and very major error rate (VMER) were calculated, with subsequent comparison to U.S. Food and Drug Administration (FDA) criteria (MER ≤3% and VMER with a 95% confidence interval ≤1.5–≤7.5%). Results: ResFinder 4.1 performed better than KOVER-AMR-models; however, neither tool achieved overall (V)MERs below FDA criteria. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1, MER (cumulative all antimicrobials) were: 5.1% (4.4–5.9), 4.3% (3.6–5.0), and 5.1% (4.5–5.8), respectively. MERs ≤3% were achieved for 6 (SCM) and 5 (CART) of the 11 tested antimicrobials for KOVER-AMR-models and for 9/13 antimicrobials tested with ResFinder 4.1. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1 cumulative VMERs were: 26% (24–28), 29% (27–31), and 11% (9.2–12). VMERs with a 95% CI ≤ 1.5–≤7.5 were only achieved for 4/13 tested antimicrobials with ResFinder 4.1. Discussion: In this study, whole genome sequence-antimicrobial susceptibility testing phenotype prediction tools KOVER-AMR and ResFinder 4.1 did not meet the FDA criteria needed for clinical diagnostic use in 517 E. coli clinical isolates from Dutch routine care. The tested tools should be further improved before they can be used for clinical decision making.
KW - E. coli
KW - External validation
KW - KOVER-AMR
KW - ResFinder
KW - WGS-AST
UR - http://www.scopus.com/inward/record.url?scp=85137624968&partnerID=8YFLogxK
U2 - 10.1016/j.cmi.2022.05.024
DO - 10.1016/j.cmi.2022.05.024
M3 - Article
C2 - 35662642
SN - 1198-743X
VL - 28
SP - 1465
EP - 1470
JO - Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases
JF - Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases
IS - 11
ER -