Impact of Survey Administration Mode on the Results of a Health-Related Discrete Choice Experiment: Online and Paper Comparison

BACKGROUND: Electronic data collection is increasingly being used for discrete choice experiments (DCEs).

OBJECTIVES: To study whether paper or electronic administration results in measurement effects.

METHODS: Respondents were drawn from the same sample frame (an Internet panel) and completed a nearly identical DCE survey either online or on paper during the same period. A DCE on preferences for basic health insurance served as a case study. We used panel mixed logit models for the analysis.

RESULTS: In total, 898 respondents completed the survey: 533 respondents completed the survey online, whereas 365 respondents returned the paper survey. There were no significant differences with respect to sociodemographic characteristics between the respondents in both samples. The median response time was shorter for the online sample than for the paper sample, and a smaller proportion of respondents from the online sample were satisfied with the number of choice sets. Although some willingness- to-pay estimates were higher for the online sample, the elicited preferences for basic health insurance characteristics were similar between both modes of administration.

CONCLUSIONS: We find no indication that online surveys yield inferior results compared with paper-based surveys, whereas the price per respondent is lower for online surveys. Researchers might want to include fewer choice sets per respondent when collecting DCE data online. Because our findings are based on a nonrandomized DCE that covers one health domain only, research in other domains is needed to support our findings.