This study evaluates irrigation infrastructure rehabilitation in Armenia. The study separately examines the impacts of tertiary canals and other large infrastructure such as main canals and pumping stations. The study also explores complementarities these types of infrastructure have with each other as well as with other activities included in the Armenia compact.
Although a random assignment design is considered the most rigorous evaluation approach and may have been feasible in this context, randomly selecting which tertiary canals would be rehabilitated was not done. Communities had to first apply to be considered for inclusion, and then canals were selected based on other factors, particularly engineering considerations and projected economic rates of return. Instead, the study uses a comparison group design. Under this approach, tertiary canals for which rehabilitation is planned will be matched to other canals sharing similar geography, pre-rehabilitation conditions, and where similar crops are grown. Examining how outcomes change for farmers in the comparison group, whose canals were not rehabilitated, will inform us about how those outcomes would have changed in the absence of the rehabilitation efforts.
Random assignment was also not possible for evaluating the large infrastructure projects. Moreover, there are too few pumping stations, gravity schemes, main canals, and drainage systems to evaluate any of those types of infrastructure separately. Thus, the evaluation uses a matched comparison group design to see whether there are impacts on communities in which any of these types of infrastructure were rehabilitated compared to those in which none was.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Anonymized dataset for public distribution
Infrastructure was rehabilitated throughout rural Armenia.
Unit of Analysis
A total of 2,990 farmers were originally interviewed across 175 communities. For the follow-up survey two communities (one treatment and one comparison) were dropped due to gunfire from across the Armenian border with Azerbaijan. Of the 173 that remain, 89 of these communities are in the tertiary canal treatment group and 77 communities are in the comparison group. The remaining nine communities will be excluded from the tertiary canal evaluation because their canals were ultimately not rehabilitated, as described previously, but we collected follow-up data on them for use in the large infrastructure evaluation. For the large infrastructure evaluation, 107 communities are in the treatment group and 66 in comparison.
Producers and sponsors
Authoring entity/Primary investigators
Mathematica Policy Research
Millennium Challenge Corporation
The sample frame for the TCS comprises the farming households served by the rehabilitated tertiary canals and the matched tertiary comparison group. Prior to the baseline TCS, the survey team worked with village mayors to identify the farmers served by each tertiary canal. Fifteen farmers were selected for interviews in most treatment communities, with rare deviations if a sampled farmer did not show up. Twenty farmers were interviewed in each comparison community. The larger number of respondents in tertiary comparison communities was to allow some cushion in case a few of the comparison group farmers were appreciably dissimilar to the associated treatment group farmers.
The large infrastructure sample should be considered a sample of convenience, that is, not statistically representative. The large infrastructure evaluation was requested and designed after the TCS baseline had already been fielded, and it was too late to commission a dedicated household survey. Instead, the large infrastructure evaluation uses the TCS data, which has a good mix of communities that were and were not served by rehabilitated large infrastructure, but it is not a representative sample of such communities.
Eighty-three percent of the baseline sample was successfully re-interviewed for the follow-up survey in the 173 communities in which interviews were attempted.
We will next use the estimated propensity scores to reweight the comparison group communities to be observably similar to the treatment communities. Separate weights will be constructed for the tertiary canal and large infrastructure analyses. Each comparison community will be assigned a weight of pc/(1 - pc). Intuitively, tertiary comparison communities that look most similar to treatment communities (pc closer to 1) receive more weight than communities that look less similar (pc closer to 0). Each treatment community will be assigned a weight of 1. These weights are constructed so that we are estimating the average impact of treatment for the communities in the treatment group.
Dates of Data Collection (YYYY/MM/DD)
Type of Research Instrument
The primary data source will be a household survey tailored to this impact evaluation, the Tertiary Canal Survey. The TCS is modeled closely after the survey used for the Water-to-Market impact evaluation, the Farming Practices Survey (FPS), and was fielded by the same survey team led by AREG. As with the FPS, the key outcomes of interest from the TCS include crops cultivated, crop production, agricultural profit, household income, and poverty. The TCS also features questions about reliability and quality of irrigation water. We conducted two rounds of the TCS.
Fortson, Kenneth, Joanne Lee, Madeline Young. 2014. Evaluation of the Irrigation Infrastructure Activity in Armenia: Design Report.