From local to global: External validity in a fertility natural experiment

Type Working Paper - IZA Discussion Papers
Title From local to global: External validity in a fertility natural experiment
Author(s)
Issue 9300
Publication (Day/Month/Year) 2015
URL https://www.econstor.eu/bitstream/10419/120951/1/dp9300.pdf
Abstract
Experimental evidence on a range of interventions in developing countries is accumulating
rapidly. Is it possible to extrapolate from an experimental evidence base to other locations of
policy interest (from “reference” to “target” sites)? And which factors determine the accuracy
of such an extrapolation? We investigate applying the Angrist and Evans (1998) natural
experiment (the effect of boy-boy or girl-girl as the first two children on incremental fertility
and mothers’ labor force participation) to data from International IPUMS on 166 country-year
censuses. We define the external validity function with extrapolation error depending on
covariate differences between reference and target locations, and find that smaller
differences in geography, education, calendar year, and mothers’ labor force participation
lead to lower extrapolation error. As experimental evidence accumulates, out-of-sample
extrapolation error does not systematically approach zero if the available evidence base is
naïvely extrapolated, but does if the external validity function is used to select the most
appropriate reference context for a given target (although absolute error remains meaningful
relative to the magnitude of the treatment effect). We also investigate where to locate
experiments and the decision problem associated with extrapolating from existing evidence
rather than running a new experiment at a target site.

Related studies

»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»