Many more impact evaluations could be done, and at
lower unit cost, if evaluators could avoid the need for
baseline data using objective socio-economic surveys and
rely instead on retrospective subjective questions on how
outcomes have changed, asked post-intervention. But
would the results be reliable? This paper tests a rapidappraisal,
“shoestring,” method using subjective recall for
welfare changes. The recall data were collected at the end
of a full-scale evaluation of a large poor-area development
This paper is a product of the Director’s office, Development Research Group. It is part of a larger effort by the World
Bank to provide open access to its research and make a contribution to development policy discussions around the world.
Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted
at mravallion@worldbank.org.
program in China. Qualitative recalls of how living
standards have changed are found to provide only weak
and biased signals of the changes in consumption as
measured from contemporaneous surveys. Importantly,
the shoestring method was unable to correct for the
selective placement of the program favoring poor villages.
The results of this case study are not encouraging for
future applications of the shoestring method, although
similar tests are needed in other settings.