According to the national Demographic and Health Surveys (DHS), between 10% and 90% of women in poorer countries agree that domestic violence against women is justified. Such wide variation in women’s responses raises concerns about the comparability of the underlying attitudinal question, and the validity of comparative research that assumes comparability. Using 67 DHS conducted in 48 countries between 1995 and 2007, we explored whether cross-national variation in women’s affirmative responses to this question were explained by cross-national variation in survey design, socioeconomic conditions, or both. Variation in survey design had considerable predictive power and accounted for a non-trivial amount of the variance in women’s affirmative responses. Including the survey-design variables also modestly altered associations of the outcome with structural variables of interest in comparative studies. These findings offer insights about future analyses and data collections on women’s attitudes about domestic violence against women.