There is an increasing interest in the use of stated preference methods to understand the preferences individuals place on health and health care. There is also a growing interest understanding the heterogeneity in their preferences. As a consequence, stated preference studies frequently consider models that capture either or both observed and unobserved preference heterogeneity. A popular preliminary investigation into heterogeneity involves split-sample analysis to draw comparisons across subgroups e.g., comparing patients with clinicians, or older patients with younger. In fixed-effects models, the constant variables (the individuals’ characteristics) remain stable of choice sets and therefore only enter the choice model when interacted with attributes and/or levels with variance. However, subgroups of respondents may differ on multiple variables that may not easily be implemented with interaction terms. This paper presents an overview of methods for matching and balancing samples in subgroup analysis and an example of how unweighted comparisons may produce erroneous conclusions regarding the degree of heterogeneity in preferences. We illustrate the issue with a synthetic dataset to explore methods for matching subgroups before and within simple choice models. Our results show that entropy balancing and propensity score matching could be more appropriate than analyses using unmatched preference data when heterogeneity is driven by multiple factors. The paper concludes with a discussion of when matching and weighting may and may not be required in healthcare decision making.
