This is the analysis plan for the impact evaluation for the Millennium Villages Project in Northern Ghana. The
purpose of the plan is to set out more specific details about how the team intendsto analyse the quantitative
datasets to estimate the impact of the intervention in 2016/17. This document builds on the earlier Initial
Design Document (IDD),
1 which set out the overarching design and the methodology for data collection and
analysis. The impact analysis is divided into a confirmatory component, directed to assess the achievement
of the Millennium Development Goal (MDG) targets using hypotheses testing and methods of causal
inference, and an exploratory component, directed to assess impact on non-MDG targets and on the causes
for success and failure in achieving some of the targets using a wider range of methods of causal inference
and directed to formulate hypotheses that could be rigorously tested in the future. We summarise here the
main elements of the plan, as follows:
 We will assess the impact of the intervention on all official MDG targets that available data allow.
 We will analyse results separately for the following sub-groups: male and female beneficiaries; residents
of Builsa and West Mamprusi districts (as of baseline administrative subdivision); and near and far control
communities (as of baseline sampling stratification by distance).
 Multiple testing of MDG impacts will be corrected by false discovery rate.
 Participation in specific programme activities and targeting are analysed.
 Impacts will be estimated by the double robust method combining difference-in-differences (DD) and
propensity score matching (PSM). DD and PSM will be combined using inverse probability weighting using
regression analysis.
 We will estimate average treatment effects (ATE) rather than average treatment effects on the treated
(ATT) in our confirmatory analysis, though the exploratory analysis will investigate impacts on specific
population groups.
 Following Angrist and Pischke (2009) we will present impact estimates of: a) one of the equivalent DD
fixed effect models illustrated in this document, and b) the DD lagged outcome model (also known as
ANCOVA) as upper and lower bounds, respectively, of the true DD effect.
 Programme participation will be modelled differently for each observation-specific outcome using logit
models.
 When data are available for more than two periods we will present estimates of average effects over the
whole period as well as effects specific to each period.
 We will conduct a number of robustness checks to assess the validity of the results against the following
threats to validity: seasonality; migration and attrition; differential trends in the outcomes; serial
correlation; selection on the unobservables; and covariate shocks.
 We will analyse spillover effects exploiting the stratification of control villages by distance made at the
sampling stage, and by using indices of social distance constructed from the social network module of the
household questionnaire. We will analyse the impact of the intervention on a number of non-MDG outcomes that capture important
domains of living standards not covered by official MDGs.
 Quantitative and qualitative teams will cooperate in formulating theories of change for specific
interventions in order to assess the impact of the programme along the causal chain.
 The qualitative team will investigate in the field main anomalies found in the analysis of the data.
 We will assess changes in returns to factors produced by the intervention by structural modelling through
an application of the Oaxaca decomposition method.
 We will test the presence of behaviours that are consistent with poverty trap models, in particular the
impact of wage expectations on educational choices and the impact of time preferences on savings and
insurance uptake.
 We will review and formulate methods for testing dynamic poverty traps that we will use once the full
panel data are available.