Water and Sanitation 2011-2013
El Salvador, 2011 - 2013
Get MicrodataIdentification
SLV_2011_MCC-WS_v01_M
Water and Sanitation 2011-2013
| Name | Country code |
|---|---|
| El Salvador | SLV |
Independent Impact Evaluation
The benefits of the water and sanitation sub activity will be measured using a rigorous quasi-experimental impact evaluation methodology. An impact evaluation is a study that measures the changes in outcomes that measure aspects of wellbeing which can be attributed to a specific intervention. Impact evaluations require a credible and rigorously defined counterfactual, which estimates what would have happened to the beneficiaries absent the project. Estimated impacts, when contrasted with total related costs, provide an assessment of the intervention's cost-effectiveness.
The evaluator divided the primary evaluation questions in different categories: welfare indicators, coping cost in cash and time, health, education, reliability and quality of service, spillover effects. In addition, we allow for differential impacts in gender and social groups for these main outcomes.
Household welfare
o Do water and sanitation infrastructure investments increase household expenditure or income? What factors might explain the impact (or lack of impact) in this area?
o What are the consequences of water and sanitation investments for expenditure patterns?
Coping costs and cash expenditure on water
o Do water and sanitation interventions reduce coping costs? What factors might explain the impact (or lack of impact) in this area?
o Do they reduce cash expenditures on water and on sanitation services? What factors might explain the impact (or lack of impact) in this area?
Health
o Do water and sanitation interventions reduce incidence of diarrheal illness?
o What factors (hygiene behavior, source and household-level water quality, household source choice) might explain the impact (or lack of impact) in this area?
Education
o Do water and sanitation interventions increase school enrollment among children aged 7 to 12? And children age 6 to 18? What factors might explain the impact (or lack of impact) in this area?
o Do water and sanitation interventions increase school attendance among children aged 7 to 12? And children age 6 to 18? What factors might explain the impact (or lack of impact) in this area?
Service, use, and sustainability
o Were the water and sanitation projects implemented according to plan?
o Are the results from the activity expected to be sustained over time?
o Did the MCC investment reach intended/unintended beneficiaries?
Gender and social exclusion
o Do the effect on health, education and access of water and sanitation interventions differ by gender or by expenditure levels (initial conditions)?
o What factors (hygiene behavior, source and household-level water quality, household source choice) might explain the impact (or lack of impact) in a specific subpopulation?
The key to measuring the impacts caused by the water and sanitation interventions is to compare conditions with the interventions to conditions that would have prevailed without them. The counterfactual state is not naturally observable - we can never know what change would have occurred in program participants (the treatment group) if the program were not implemented. As it was not possible to apply randomization in the selection of water and sanitation projects in this case, the benefits of the water and sanitation projects will be measured with a rigorous quasi-experimental design that incorporates matching, pre- and post-implementation data collection, difference-in-difference estimation, and econometric analysis to estimate the counterfactual and address selection and other biases. This requires selecting a comparison group-households that are observationally similar to beneficiary households but do not participate in the program-and observing both sets of households before and after the program is implemented.
Matching represents a credible non-experimental option for identifying comparison groups. The evaluator uses propensity score matching (PSM) using data from the 2007 census to match the treatment communities to comparable communities before program implementation. PSM identifies comparison communities that have a similar probability of receiving the treatment and are similar to the treatment communities in terms of observable characteristics. Accordingly, they provide measures of indicators in communities that are similar except for the treatment; thus addressing selection on observables.
Sample survey data [ssd]
Census segments, households, individuals
Scope
| Topic |
|---|
| El Salvador |
| WASH |
| water |
| sanitation |
| impact evaluation |
| PSM |
| matching |
Coverage
The survey was administered in the El Salvador Northern Zone departments of Cabañas, Chalatenango, Cuscatlán, La Unión, Morazán, San Miguel and Santa Ana.
Sixty-two municipalities in the Northern Zone, classified as either “Extrema Pobreza Moderada” or “Extrema Pobreza Alta” (extreme moderate poverty or extreme high poverty, respectively) by the national poverty map, were invited to submit proposals for water and sanitation projects. To be considered eligible for the program, the proposals had to meet four criteria: (1) the municipality had to be eligible to participate, meaning there were classified as high or moderate extreme poverty; (2) both the community and municipality had to be willing to make a financial/labor commitment to the project, (3) the community had to be organized and willing to work with the municipality, and (4) the estimated cost of the project could not exceed $850 per beneficiary. After projects that did not meet the eligibility criteria were excluded, a list of 68 projects remained. These were cleared to enter the feasibility stage. Comparisons segments were selected from non-beneficiary segments that where eligible to participate taking into account the poverty map, an proxies for financial capacity of the municipality and community involvement where included in the propensity score estimation.
Producers and sponsors
| Name |
|---|
| Social Impact |
| Name |
|---|
| Millennium Challenge Corporation |
Sampling
In 2009, the evaluator recommended 18 observations per cluster and 164 communities, while adding an additional contingency -- 6 extra treatment segments and 6 extra control segments, for a total of 216 additional households in order to provide extra cushion for the loss of projects during implementation or inaccuracies in the sample frame. The final sample size for recommended for the study was 3,168, with 88 comparison and 88 treatment segments, each with 18 households.
However, in 2011, given changes to program design and required revisions to power calculations, the evaluator ultimately collected baseline data on 3,284 households, with 65 segments in both treatment and control and an average of 24-27 households per segment.
94.5% for 2012 survey
96% for 2013 survey
Survey instrument
The household level survey is administered in the departments of Cabañas, Chalatenango, Cuscatlán, La Unión, Morazán, San Miguel and Santa Ana. The survey is composed of a set of sections to characterize the water access situation of households, household demographics, consumption, income/productive activities and time allocation of women and children.
The community level survey includes 130 census segments representing 196 caseríos. The information is obtained from interviews of key informants from the communities. Key informants include health workers/promoters, members of the water boards and other community leaders.
Data collection
| Start | End |
|---|---|
| 2011 | 2011 |
| 2012 | 2012 |
| 2013 | 2013 |
| Name |
|---|
| Dirección General de Estadísticas y Censos |
Interviewing was conducted by teams of interviewers. Each interviewing team comprised of 3-4 interviewers, and a supervisor, and a driver.
The role of the supervisor was to coordinate field data collection activities, including management of the field teams, supplies and equipment, finances, maps and listings, coordinate with local authorities concerning the survey plan and make arrangements for accommodation and travel. Additionally, a chief field supervisor assigned the work to the supervisors/interviewers, spot checked work, maintained field control documents, and sent completed questionnaires and progress reports to the central office.
The team 2 coordinators for data entry and quality control that were responsible for managing the headquarter team reviewing each questionnaire, checking for missed questions, skip errors, fields incorrectly completed, and checking for inconsistencies in the data.
Access policy
Millennium Challenge Corporation
Millennium Challenge Corporation
http://data.mcc.gov/evaluations/index.php/catalog/107
Cost: None
Data Access
Millennium Challenge Corporation, 2016. Panel Survey for Impact Evaluation of the MCC Water and Sanitation Sub-activity in El Salvador 2011-2013, Public use files for [Dataset Collection Year(s)]. Date Accessed: [Date accessed]. Accessed at: [URL where data was accessed].
Contacts
| Name | |
|---|---|
| Monitoring and Evaluation Division of the Millennium Challenge Corporation | impact-eval@mcc.gov |
Metadata production
DDI_SLV_2011_MCC-WS_v02_M
| Name | Role |
|---|---|
| Millennium Challenge Corporation | Review of Metadata |
2014-09
Metadata version
Version 1 (2014-08-20)
Version 1.1 (2014-09-09): Added survey team description and survey implementer information
Version 2.0 (April 2015): Edited version based on Version 01 (DDI-MCC-SLV-WASH-2014-v01) that was done by Millennium Challenge Corporation.
Version 02 (March 2019): Edited version based on Version 1.2 (DDI-MCC-SLV-WASH-2014-v1.2) that was done by Millennium Challenge Corporation.
Sixty-two municipalities in the Northern Zone, classified as either extreme moderate poverty or extreme high poverty by the national poverty map, were invited to submit proposals for water and sanitation projects. To be considered eligible for the program, the proposals had to meet four criteria: (1) the municipality had to be classified as either extreme moderate poverty or extreme high poverty, (2) both the community and municipality had to be willing to make a financial commitment to the project, (3) the community had to be organized and willing to work with the municipality, and (4) the estimated cost of the project could not exceed $850 per beneficiary.