Using web-based and paper-based questionnaires for collecting data on fertility issues among female childhood cancer survivors: differences in response characteristics

BACKGROUND: Web-based questionnaires have become increasingly popular in health research. However, reported response rates vary and response bias may be introduced.

OBJECTIVE: The aim of this study was to evaluate whether sending a mixed invitation (paper-based together with Web-based questionnaire) rather than a Web-only invitation (Web-based questionnaire only) results in higher response and participation rates for female childhood cancer survivors filling out a questionnaire on fertility issues. In addition, differences in type of response and characteristics of the responders and nonresponders were investigated. Moreover, factors influencing preferences for either the Web- or paper-based version of the questionnaire were examined.

METHODS: This study is part of a nationwide study on reproductive function, ovarian reserve, and risk of premature menopause in female childhood cancer survivors. The Web-based version of the questionnaire was available for participants through the Internet by means of a personalized user name and password. Participants were randomly selected to receive either a mixed invitation (paper-based questionnaire together with log-in details for Web-based questionnaire, n = 137) or a Web-only invitation (log-in details only, n = 140). Furthermore, the latter group could request a paper-based version of the questionnaire by filling out a form.

RESULTS: Overall response rates were comparable in both randomization groups (83% mixed invitation group vs 89% in Web-only invitation group, P = .20). In addition, participation rates appeared not to differ (66% or 90/137, mixed invitation group vs 59% or 83/140, Web-only invitation group, P =.27). However, in the mixed invitation group, significantly more respondents filled out the paper-based questionnaire compared with the Web-only invitation group (83% or 75/90 and 65% or 54/83, respectively, P = .01). The 44 women who filled out the Web-based version of the questionnaire had a higher educational level than the 129 women who filled out the paper-based version (P = .01). Furthermore, the probability of filling out the Web-based questionnaire appeared to be greater for women who were allocated to the Web-only invitation group (OR = 2.85, 95% CI 1.31-6.21), were older (OR = 1.08, 95% CI 1.02-1.15), had a higher educational level (OR high vs low = 0.06, 95% CI 0.01-0.52), or were students (OR employed vs student = 3.25, 95% CI 1.00-10.56).

CONCLUSIONS: Although overall response as well as participation rates to both types of invitations were similar, adding a paper version of a questionnaire to a Web-only invitation resulted in more respondents filling out the paper-based version. In addition, women who were older, had a higher level of education, or were students, were more likely to have filled out the Web-based version of the questionnaire. Given the many advantages of Web-based over paper-based questionnaires, researchers should strongly consider using Web-based questionnaires, although possible response bias when using these types of questionnaires should be taken into account.

TRIAL REGISTRATION: Nederlands Trial Register NTR2922; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=2922 (Archived by WebCite at http://www.webcitation.org/5zRRdMrDv).