PURPOSE: In nonexperimental comparative effectiveness research, restricting analysis to subjects with better overlap of covariate distributions, hence greater treatment equipoise, helps balance the groups compared and can improve validity. Three alternative approaches, derived from different perspectives, implement restriction by trimming observations in the tails of the propensity score (PS). Across approaches, we compared the relationships between the overlap in treatment-specific PS distributions and the size of the balanced study population after trimming.
METHODS: The three trimming approaches considered were absolute trimming to the range 0.1 less than PS less than 0.9, asymmetric trimming to include subjects in both treatment groups with PS above the 5th percentile of the distribution in the target group and below the 95th percentile in the comparison group, and restriction to preference score values between 0.3 and 0.7. Comparisons of approaches used simulated PSs from beta distributions and two example studies.
RESULTS: The magnitude of the C-statistic strongly predicted (R2 ≥.95) the percent of the balanced study population remaining. The balanced study population was largest under trimming at absolute PS levels, unless the target treatment was uncommon. Fewer than half of original study subjects remained after preference score trimming if C≥.80 and after asymmetric trimming if C≥.85. In examples, trimming improved the precision of estimated risk differences and identified apparent treatment effect heterogeneity in the PS tails where covariate balance was limited. Relative amounts of trimming in examples reflected the simulation results.
CONCLUSIONS: Study populations with high PS C-statistics include only small percentages of subjects in whom valid treatment effects are confidently expected.
