Alatona Irrigation 2009-2012

The key distinction between impact evaluation and other monitoring and evaluation techniques is that impact evaluation seeks to isolate the causal relationship between interventions and the welfare or wellbeing of beneficiaries. Given the objective of MCC to enhance economic growth, wellbeing will generally be captured by household consumption, assets, or income, at the household level, but occasionally at the individual level with measures such as assets or income to investigate gender effects. Since there are many factors influencing households' consumption, income, and wellbeing in a given year, including multiple projects within an MCC Compact, often a simple before-and-after comparison can lead to a misleading or incorrect assessment of project impacts. The challenge of impact evaluation, therefore, is to identify suitable comparison groups to compare with beneficiaries. Randomization is considered the gold standard, since it is the best tool available to address confounding observable and unobservable factors in a research design by ensuring their balance across treatment and control groups. However, randomization was not feasible for the PAP evaluation: legally the individuals living on the land turned into irrigated land were automatically beneficiaries. Therefore, the evaluation team is using PSM to evaluate the total impact of the AIP on the PAPs versus individuals and households living in the same general geographic area who had similar characteristics to the PAP prior to the program but that did not directly benefit from getting land.

The use of PSM relies on the untestable assumption that there are no unobservable differences between villages with PAPs and other communities in the area of the ON. If unobserved factors influence the rate of adoption between the treatment and comparison groups, then this could bias the outcome indicators. This report provides a number of analyses to see if the estimates are robust to changes in how the analysis is done, and uses both baseline and follow up data to give us more confidence in the estimates. Nevertheless, it is important that readers keep in mind that we have less overall confidence in these impact estimates than if the project had been evaluated using the RCT method.

There are some aspects of the AIP which are not evaluated through rigorous impact evaluation, such as the Niono-Goma road and the improvements to the main water conveyance system. All residents living near the road will have benefited from the program, not just the PAPs. For the water conveyance system, there are many more beneficiaries of this component of the project (all farmers with irrigated land in the ON), and this impact evaluation will not, and was not designed to, provide any estimate of those project impacts.