Does the same question asked of the same population yield
the same answer in face-to-face interviews when other parts
of the questionnaire are altered? If not, what would be
the implications for proxy-based poverty measurement?
Relying on a randomized household survey experiment
implemented in Malawi, this study finds that observationally
equivalent as well as same households answer the
same questions differently when interviewed with a short
questionnaire versus the longer counterpart that, in a prior
survey round, would have informed the prediction model
for a proxy-based poverty measurement exercise. The analysis
yields statistically significant differences in reporting
between the short and long questionnaires across all topics
and types of questions. The reporting differences result
in significantly different predicted poverty rates and Gini
coefficients. While the difference in predictions ranges from
approximately 3 to 7 percentage points depending on the
model specification, restricting the proxies to those collected
prior the variation in questionnaire design, namely
demographic variables from the household roster and
location fixed effects, leads to same predictions in both
samples. The findings emphasize the need for further methodological
research, and suggest that short questionnaires
designed for proxy-based poverty measurement should
be piloted, prior to implementation, in parallel with the
longer questionnaire from which they have evolved. The
fact that at the median it took 25 minutes to complete
the food and non-food consumption sections in the long
questionnaire also implies that the implementation of these
sections might not be as overly costly as usually assumed.