TY - JOUR
T1 - Signal detection in VAERS and EudraVigilance using disproportionality and time to onset method and their combination
AU - Dodd, Caitlin
AU - van Holle, Lionel
AU - de Ridder, Maria
AU - Weibel, Daniel
AU - Sturkenboom, Miriam
PY - 2018/8
Y1 - 2018/8
N2 - Background: Adverse events following immunization (AEFIs) may be
monitored in spontaneous reporting systems using signal detection
algorithms which compare the frequency with which a vaccine‐AEFI
combination is reported as compared with all reports in the database.
Alternatively, a recently developed method uses distribution test of
reported time‐to‐onset (TTO) to detect vaccine safety signals with
unexpected reported temporal relationship.
Objectives: To compare performance of the Empirical Bayes Geometric
Mean (EBGM), the TTO method, and their combination in pediatric
vaccine reports in the databases (DBs) of the US Vaccine Adverse
Event Reporting System (VAERS) and EU EudraVigilance (EV) as part
of the Global Research in Pediatrics project.
Methods: Following DB conversion to a common data model, EBGM
and TTO methods were applied to all pediatric vaccine reports and
in subgroups of infants (age < 2 yr), children (2‐11 yr), and adolescents
(12‐17 yr) in VAERS, EV, and their combination. Performance of each
method was assessed using a previously published vaccine reference
set comprising 18 true positive and 113 true negative associations
to calculate area under the receiver operating characteristic curve
(AUC). The method of Pepe & Thompson (2000) was used to determine
the linear combination of EBGM and TTO which maximized
the AUC in the VAERS database.
Results: VAERS contained 1.85 million pediatric vaccine reports, in
which TTO data was missing in 8%, with highest missing percentage
in adolescents (10%). EV contained 251 770 pediatric vaccine
reports, 57% of which were missing TTO data (74% missing in adolescents).
In VAERS, the TTO method AUCs in the full pediatric population,
infants, children, and adolescents were 0.89, 0.88, 0.89, and
0.73 while EBGM AUCs for the same age groups were 0.73, 0.67,
0.72, and 0.69. Performance in EV was similar with the exception
that EBGM performed better (AUC = 0.80) than TTO (AUC = 0.70)
in adolescents. The combination of TTO and EBGM in pediatrics in
VAERS increased AUC to 0.91 from AUCs of 0.89 and 0.73 for the
TTO and EBGM components, respectively.
Conclusions: The TTO method performed better than EBGM in both
VAERS and EV in pediatric age groups with the exception of adolescents in EV. Lower performance of the TTO method in adolescents could be related to a higher percentage of missing values in that age group. The combination of EBGM and TTO methods only marginally improved performance compared with TTO method alone.
AB - Background: Adverse events following immunization (AEFIs) may be
monitored in spontaneous reporting systems using signal detection
algorithms which compare the frequency with which a vaccine‐AEFI
combination is reported as compared with all reports in the database.
Alternatively, a recently developed method uses distribution test of
reported time‐to‐onset (TTO) to detect vaccine safety signals with
unexpected reported temporal relationship.
Objectives: To compare performance of the Empirical Bayes Geometric
Mean (EBGM), the TTO method, and their combination in pediatric
vaccine reports in the databases (DBs) of the US Vaccine Adverse
Event Reporting System (VAERS) and EU EudraVigilance (EV) as part
of the Global Research in Pediatrics project.
Methods: Following DB conversion to a common data model, EBGM
and TTO methods were applied to all pediatric vaccine reports and
in subgroups of infants (age < 2 yr), children (2‐11 yr), and adolescents
(12‐17 yr) in VAERS, EV, and their combination. Performance of each
method was assessed using a previously published vaccine reference
set comprising 18 true positive and 113 true negative associations
to calculate area under the receiver operating characteristic curve
(AUC). The method of Pepe & Thompson (2000) was used to determine
the linear combination of EBGM and TTO which maximized
the AUC in the VAERS database.
Results: VAERS contained 1.85 million pediatric vaccine reports, in
which TTO data was missing in 8%, with highest missing percentage
in adolescents (10%). EV contained 251 770 pediatric vaccine
reports, 57% of which were missing TTO data (74% missing in adolescents).
In VAERS, the TTO method AUCs in the full pediatric population,
infants, children, and adolescents were 0.89, 0.88, 0.89, and
0.73 while EBGM AUCs for the same age groups were 0.73, 0.67,
0.72, and 0.69. Performance in EV was similar with the exception
that EBGM performed better (AUC = 0.80) than TTO (AUC = 0.70)
in adolescents. The combination of TTO and EBGM in pediatrics in
VAERS increased AUC to 0.91 from AUCs of 0.89 and 0.73 for the
TTO and EBGM components, respectively.
Conclusions: The TTO method performed better than EBGM in both
VAERS and EV in pediatric age groups with the exception of adolescents in EV. Lower performance of the TTO method in adolescents could be related to a higher percentage of missing values in that age group. The combination of EBGM and TTO methods only marginally improved performance compared with TTO method alone.
M3 - Meeting Abstract
SN - 1053-8569
VL - 27
SP - 251
EP - 252
JO - Pharmacoepidemiology and Drug Safety
JF - Pharmacoepidemiology and Drug Safety
IS - S2
ER -