TY - JOUR
T1 - Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes
T2 - instrument development and reliability study
AU - Devji, Tahira
AU - Carrasco-Labra, Alonso
AU - Qasim, Anila
AU - Phillips, Mark
AU - Johnston, Bradley C
AU - Devasenapathy, Niveditha
AU - Zeraatkar, Dena
AU - Bhatt, Meha
AU - Jin, Xuejing
AU - Brignardello-Petersen, Romina
AU - Urquhart, Olivia
AU - Foroutan, Farid
AU - Schandelmaier, Stefan
AU - Pardo-Hernandez, Hector
AU - Vernooij, Robin Wm
AU - Huang, Hsiaomin
AU - Rizwan, Yamna
AU - Siemieniuk, Reed
AU - Lytvyn, Lyubov
AU - Patrick, Donald L
AU - Ebrahim, Shanil
AU - Furukawa, Toshi
AU - Nesrallah, Gihad
AU - Schünemann, Holger J
AU - Bhandari, Mohit
AU - Thabane, Lehana
AU - Guyatt, Gordon H
N1 - Funding Information:
1Department of Health Research Methods, Evidence and Impact, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4L8, Canada 2Department of Community Health and Epidemiology, Dalhousie University, Halifax, NS, Canada 3Indian Institute of Public Health, Public Health Foundation of India, Gujarat, India 4School of Public Health, University of Alberta, Edmonton, AB, Canada 5Center for Evidence Based Dentistry, American Dental Association, Chicago, IL, USA 6Iberoamerican Cochrane Centre, Sant Pau Biomedical Research Institute (IIB Sant Pau), Barcelona, Spain 7CIBER de Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain 8Department of Research, Comprehensive Cancer Organisation, Utrecht, Netherlands 9Department of Orthopedic Surgery, University of Michigan, Ann Arbor, MI, USA 10Department of Molecular and Cellular Biology, University of Guelph, Guelph, ON, Canada 11Department of Health Services, University of Washington, Seattle, WA, USA 12Department of Health Promotion and Human Behaviour, School of Public Health, Kyoto University Graduate School of Medicine, Kyoto, Japan 13Nephrology Program, Humber River Regional Hospital, Toronto, ON, Canada 14Division of Nephrology, University of Western Ontario, London, ON, Canada 15Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, ON, Canada 16Department of Medicine, McMaster University, Hamilton, ON, Canada 17Department of Surgery, McMaster University, Hamilton, ON, Canada The Credibility instrument for judging the trustworthiness of minimal important difference estimates, authored by Devji et al, is the copyright of McMaster University (copyright 2018, McMaster University). The Credibility instrument for judging the trustworthiness of minimal important difference estimates has been provided under license from McMaster University and must not be copied, distributed, or used in any way without the prior written consent of McMaster University. Contact the McMaster Industry Liaison Office at McMaster University ([email protected]) for licensing details. Contributors: TD and AC-L are joint first authors. TD, AC-L, GHG, BCJ, GN, and SE conceived the study idea; TD, AC-L, AQ, MP, and GHG led the development of the credibility instrument; TD, AC-L, AQ, MP, ND, DZ, MB, XJ, RB-P, OU, FF, SS, HP-H, RWMV, HH, YR, RS, and LL extracted data and assessed the credibility of MIDs in our inventory for the reliability analyses; TD and AC-L wrote the first draft of the manuscript; TD, AC-L, GHG, AQ, MP, ND, DZ, RB-P, OU, SS, HP-H, RWMV, LL, BCJ, DLP, SE, TF, GN, HJS, MB, and LT interpreted the data analysis and critically revised the manuscript. TD and AC-L are the guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. Funding: This project was funded by the Canadian Institutes of Health Research (CIHR), Knowledge Synthesis (grant No KRS138214). The views expressed in this work are those of the authors and not necessarily represent the views of the CIHR or the Canadian government.
Publisher Copyright:
© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2020/6/4
Y1 - 2020/6/4
N2 - OBJECTIVE: To develop an instrument to evaluate the credibility of anchor based minimal important differences (MIDs) for outcome measures reported by patients, and to assess the reliability of the instrument.DESIGN: Instrument development and reliability study.DATA SOURCES: Initial criteria were developed for evaluating the credibility of anchor based MIDs based on a literature review (Medline, Embase, CINAHL, and PsycInfo databases) and the experience of the authors in the methodology for estimation of MIDs. Iterative discussions by the team and pilot testing with experts and potential users facilitated the development of the final instrument.PARTICIPANTS: With the newly developed instrument, pairs of masters, doctoral, or postdoctoral students with a background in health research methodology independently evaluated the credibility of a sample of MID estimates.MAIN OUTCOME MEASURES: Core credibility criteria applicable to all anchor types, additional criteria for transition rating anchors, and inter-rater reliability coefficients were determined.RESULTS: The credibility instrument has five core criteria: the anchor is rated by the patient; the anchor is interpretable and relevant to the patient; the MID estimate is precise; the correlation between the anchor and the outcome measure reported by the patient is satisfactory; and the authors select a threshold on the anchor that reflects a small but important difference. The additional criteria for transition rating anchors are: the time elapsed between baseline and follow-up measurement for estimation of the MID is optimal; and the correlations of the transition rating with the baseline, follow-up, and change score in the patient reported outcome measures are satisfactory. Inter-rater reliability coefficients (ĸ) for the core criteria and for one item from the additional criteria ranged from 0.70 to 0.94. Reporting issues prevented the evaluation of the reliability of the three other additional criteria for the transition rating anchors.CONCLUSIONS: Researchers, clinicians, and healthcare policy decision makers can consider using this instrument to evaluate the design, conduct, and analysis of studies estimating anchor based minimal important differences.
AB - OBJECTIVE: To develop an instrument to evaluate the credibility of anchor based minimal important differences (MIDs) for outcome measures reported by patients, and to assess the reliability of the instrument.DESIGN: Instrument development and reliability study.DATA SOURCES: Initial criteria were developed for evaluating the credibility of anchor based MIDs based on a literature review (Medline, Embase, CINAHL, and PsycInfo databases) and the experience of the authors in the methodology for estimation of MIDs. Iterative discussions by the team and pilot testing with experts and potential users facilitated the development of the final instrument.PARTICIPANTS: With the newly developed instrument, pairs of masters, doctoral, or postdoctoral students with a background in health research methodology independently evaluated the credibility of a sample of MID estimates.MAIN OUTCOME MEASURES: Core credibility criteria applicable to all anchor types, additional criteria for transition rating anchors, and inter-rater reliability coefficients were determined.RESULTS: The credibility instrument has five core criteria: the anchor is rated by the patient; the anchor is interpretable and relevant to the patient; the MID estimate is precise; the correlation between the anchor and the outcome measure reported by the patient is satisfactory; and the authors select a threshold on the anchor that reflects a small but important difference. The additional criteria for transition rating anchors are: the time elapsed between baseline and follow-up measurement for estimation of the MID is optimal; and the correlations of the transition rating with the baseline, follow-up, and change score in the patient reported outcome measures are satisfactory. Inter-rater reliability coefficients (ĸ) for the core criteria and for one item from the additional criteria ranged from 0.70 to 0.94. Reporting issues prevented the evaluation of the reliability of the three other additional criteria for the transition rating anchors.CONCLUSIONS: Researchers, clinicians, and healthcare policy decision makers can consider using this instrument to evaluate the design, conduct, and analysis of studies estimating anchor based minimal important differences.
KW - Health Status Indicators
KW - Humans
KW - Patient Reported Outcome Measures
KW - Quality of Life
KW - Reproducibility of Results
KW - Research Design
KW - Surveys and Questionnaires
UR - https://www.scopus.com/pages/publications/85086008409
U2 - 10.1136/bmj.m1714
DO - 10.1136/bmj.m1714
M3 - Article
C2 - 32499297
SN - 1756-1833
VL - 369
SP - m1714
JO - BMJ (Clinical research ed.)
JF - BMJ (Clinical research ed.)
M1 - m1714
ER -