OBJECTIVES: There is continued interest in using stated preference methods, particularly discrete choice experiments (DCEs), to understand preferences for health and healthcare. Stated preference methods are underpinned by economic theories which make assumptions about individuals’ choicemaking behaviour. To investigate violations, various techniques have been used to reveal stated preference survey respondents’ cognitive processes. However, in trying to measure decision making do researchers change respondents’ decisions?
METHODS: This study used data collected from a DCE designed to elicit preferences for a national breast screening programme described by three attributes: probability of detecting a cancer; risk of unnecessary follow-up; and cost of screening. The experiment was conducted in three settings: a ‘traditional’ online quantitative approach (n=1000); face-to-face think-aloud interviews (n=19); and a laboratory-based eye-tracking study (n=40). Choice data were analysed using heteroskedastic conditional logit models to compare across experiment settings.
RESULTS: There were statistically significant differences in the valuations derived from the three experimental settings. Compared with the internet panel, the attribute ‘cost of screening’ had less weight (P<0.05) for the think-aloud sample whereas the ‘risk of unnecessary follow-up’ had more weight for the eye-tracking sample (P<0.05). Potential reasons why valuations derived from the three experimental settings may have differed include: 1) respondents may have sought to conform to ‘social norms’ when researchers were present; 2) the act of being watched changed respondents’ attention (the ‘Hawthorne Effect’); and/or 3) the experiments attracted different types of women with different preferences.
CONCLUSIONS: Novel experimental methods alongside stated preference studies can be useful for illuminating certain phenomena, but researchers should be aware that these approaches may alter respondents’ decision-making processes or attract respondents with different characteristics both of which could result in different choices and differences in the valuations derived.
