Sustainable Water Supply and Sanitation Project Impact Evaluation 2019, Endline Survey
Nicaragua, 2019
Get MicrodataIdentification
NIC_2019_PROSASRIE-EL_v01_M
Sustainable Water Supply and Sanitation Project Impact Evaluation 2019, Endline Survey
Evaluación de Impacto del Proyecto de Sostenibilidad del Sector de Agua y Saneamiento Rural (PROSASR)
| Name | Country code |
|---|---|
| Nicaragua | NIC |
Impact Evaluation Survey
The objective of the impact evaluation is to assess if quality technical assistance provided by municipalities, as part of Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR, by its acronym in Spanish), translates into better functionality and durability of water supply and sanitation (WSS) systems in rural Nicaragua. A secondary objective is to explore if, over the long run, the provision of quality and sustained service enhancement translates into better health outcomes for beneficiary populations within these communities.
This work was conducted in collaboration with the Fondo de Inversión Social y Emergencias (FISE, by its acronym in Spanish) of the Nicaraguan Government. Funding from the World Bank Strategic Impact Evaluation Fund (SIEF) helped in the design, in the evaluation, in the collection of data, and in helping support research activities.
The results from the baseline study, conducted in 2015/2016 were published (NIC_2015-2016_PROSASRIE-BL_v01_M) under a separate study. A second round of end-line data-collection was conducted in March to November 2019. The data published here includes the end-line survey data. The report (found under the 'documentation' tab) includes impact results from 2015 to 2019. Future analysis, using data from the Rural Water and Sanitation information system (SIASAR), as well as a potential new data collection effort in two years, could elucidate the long-term effects of this intervention.
The end-line survey-data collection was gathered at the following three levels:
First, surveys to the Water and Sanitation Municipality Units (UMAS, by its acronym in Spanish) which examine, among other items, their institutional capacity in terms of: their human resources, their availability of equipment for water quality monitoring, their access to a computer, to vehicles, whether they have informative printed material, whether they have available assigned annual budgets, availability of funds for travel expenses and fuel, and their access to internet services. They also capture overall level of support of UMAS to communities.
Second, surveys to the Water and Sanitation Committees (CAPS, by its acronym in Spanish), which are formal or informal institutions that manage, operate, maintain, and repair water distribution systems at the community-level. The surveys measure progress in terms of formal operation, adequate tariffs for water supplied, financial stability, operation and maintenance, proper protection of the water source, and cover details on the process of legalization. These surveys also measure the capacity and functionality of water systems.
Finally, the household surveys offer a detailed assessment of WASH services, and the level of satisfaction with the service provided by the CAPS, at the household level. They measure, from the household perspective, any improvements in water supply in terms of quality, continuity, access, whether community-level interaction with the CAPS has increased, and any knock-on health implications as a result of possible increases in the quality of water.
Sample survey data [ssd]
The surveys target 3 units of analysis:
-
A survey to the UMAS. These are municipal-level representatives responsible for providing technical assistance to the CAPS.
-
A survey directed at the CAPS. This consisted of a survey addressed at covering their institutional and financial solidity, and the functionality of the water systems they supervise.
-
A household survey - addressed to the head of the household
Version
-v01
Scope
-
The scope of the PAT/UMAS survey includes:
a) basic information on the PAT - what kind of organization it is, how many communities it has assigned to it, how many have solicited and received support in the past years
b) the financial, human and logistical resources available to the PAT, including whether it has budget and vehicles to travel to the communities
c) their perspectives on the trainings they receive and would like to receive
d) their perspectives on the needs of the CAPS from the perspective of the UMAS -
The scope of the CAPS survey includes:
a) The community: how many households there are in the community, the main language and ethnicities, schools and health center WASH facilities, sanitation facilities coverage in the community, hygiene in the households and rubbish collection.
b) The system: this contains questions on the state of the water system infrastructure - identifying its different components such as the source, tank, distribution network and whether there is water treatment in the system. It also assesses water quality and accessibility as well as the type of system and the number of households it serves.
c) The service provider: this contains questions on the CAPS or committee's legal status, whether it is formalized, how representative it is, the tariffs they charge, whether they have micrometers installed, and a variety of economic information, their operation and maintenance activities, environmental protection and sources of financing for the initial construction of the system.
- Household surveys: these contained basic demographic questions on the number of members of the household, age, marital status, ethnicity, education and income of the head of the household. It also contains information on any health issues suffered in the past week such as diarrhea. In the next section, there are a series of questions on the system and the water service such as how much they pay for the tariff, the quality of the water they receive and quantity they use and have available to them. The survey also asks questions on the household's interactions with the CAPS and whether they received any trainings from them or would like to receive any. Finally, it asks about trash disposal and the conditions on route to the household.
Coverage
Rural areas only
The intervention focused on rural communities in the country. The sample of selected communities was restricted to: include those communities which had a minimum level of infrastructure, exclude those communities that were the subject of other interventions, or that were too small or big according total number of households. Moreover, only those municipalities that had at least four communities were considered.
These steps yielded a sample consisting of 100 municipalities and 1,600 communities. Out of this sample, 75 municipalities were randomly selected. Within each of these municipalities, between two and four communities were randomly selected, with an equal number allocated to treatment and control communities.
The starting point was a universe of 6,862 communities across 148 municipalities that were in the SIASAR database and constituted almost the entire country. Community eligibility criteria were introduced in order to focus the evaluation on communities that would most likely improve their indicators with the intervention, given their initial status.
The end-line data includes: household data of 4527 households in 299 communities, data at the municipality level; administered to 69 UMAS, and 226 surveys administered to CAPS. The intervention covered 76 of the 153 municipalities in Nicaragua.
Producers and sponsors
| Name |
|---|
| The World Bank |
| Name | Affiliation | Role |
|---|---|---|
| Christian Borja Vega | The World Bank | Primary Investigator |
| Pavel Luengas Sierra | The World Bank | |
| Jonthan Grabinsky | The World Bank |
| Name | Role |
|---|---|
| Strategic Impact Evaluation Fund - The World Bank | Main Funder of the Impact Evaluation |
| Fondo de Inversión Social de Emergencia | PROSASR implementing agency, in kind contributions through mobilization and facilitation of government contacts |
| Spanish Fund for Latin America and the Caribbean | Contributed through the SIASAR initiative to data quality monitoring |
Sampling
SAMPLE SIZE: The size of the eventual sample was 300 communities - 150 assigned to treatment and 150 assigned to control.
SELECTION PROCESS: The starting point was a universe of 6,862 communities across 148 municipalities that were in the SIASAR database and constituted almost the entire country. Community eligibility criteria were introduced in order to focus the evaluation on communities that would most likely improve their indicators with the intervention, given their initial status. The criteria for the selection process is described in more detail in the study report found under the ‘Documentation’ tab.
STRATIFICATION AND STAGES OF SAMPLE SELECTION: The sample of 300 communities was selected, across 75 municipalities and within each municipality the 4 eligible communities were randomized to either treatment or control (2 allocated to treatment and 2 allocated to control). This resulted in 150 treatment communities and 150 control communities evenly distributed across the 75 municipalities. Within the 150 treatment and 150 control communities, 100 communities come from 50 municipalities classified as "less-poor" and 50 communities across 25 municipalities that are "poor"
Municipalities were stratified (poor, less poor) and randomly ordered; the first 50 less poor and the first 25 poor municipalities in each list were selected for inclusion in the evaluation. Within each selected municipality all eligible communities were randomly ordered. The first two communities became control communities and were automatically enrolled in the evaluation. The remaining randomly ordered communities (3 through to n) were all assigned to receive the treatment from the municipality; two communities from the treatment list (all communities below 3 in the original random ordering) were randomly selected to be enrolled in the evaluation (data collection) in a manner consistent with the random ordering described.
LEVELS OF REPRESENTATION: As the original universe was from all the communities in the country registered in SIASAR (nearly every community nationally), and we then selected randomly half the municipalities in the country, our sample has a good level of representation. We also checked for balance on distribution across the treatment and control groups of the communities in the Pacific Coast, Central Region and Atlantic or Caribbean Coast.
During baseline fieldwork, owing to logistical issues and interviewer safety, the evaluation team had to make minor adjustment to the community list. The team replaced (1) one municipality (four communities), (2) two communities from one municipality with two from another, and (3) one community for another. For this reason, the final sample after baseline work had 76 municipalities instead of the 75 planned. These replacements and substitutions used randomization to ensure that the initial experiment design was preserved.
Household surveys:
Total number of HH surveys requested in sample: 5000
Total number of HH surveys collected in the field at baseline: 4850
Total number of HH surveys collected in the field at endline: 4527
Percentage of HH that participated at endline: N/A - different households were interviewed at baseline and endline
PAT surveys:
Total number of PAT surveys required in sample: 75
Total number of PAT surveys collected in the field at baseline: 78
Total number of PAT surveys collected in the field at endline: 69
Percentage of PATs that participated at endline: 88.5%
(comment: the reason for more PAT surveys being carried out than the original sample is due to replacement communities being selected from new municipalities where PAT surveys were carried out in original and replacement municipalities)
CAPS – Community Committee surveys:
Total number of COM surveys required in sample: 300
Total number of COM surveys collected at baseline: 299
Total number of COM surveys collected at endline: 226
Percentage of COMs completed at endline: 75.6%
Data collection
| Start | End | Cycle |
|---|---|---|
| 2019-03-01 | 2019-10-31 | Endline |
Data appraisal
Not applicable because not a sample survey
Data Access
| Name | Affiliation | |
|---|---|---|
| Christian Borja Vega | The World Bank | cborjavega@worldbank.org |
Direct access
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
The World Bank Group. Nicaragua Sustainable Water Supply and Sanitation Project Impact Evaluation, Endline Survey (PROSASRIE-EL) 2019, Ref. NIC_2019_PROSASARIE-EL_v01_M. Dataset downloaded from [url] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
(c) 2019, The World Bank
Contacts
| Name | Affiliation | |
|---|---|---|
| Christian Borja Vega | World Bank | cborjavega@worldbank.org |
Metadata production
DDI_NIC_2019_PROSASRIE-EL_v01_M_WB
| Name | Affiliation | Role |
|---|---|---|
| Development Economics Data Group | The World Bank | Documentation of the DDI |
2020-07-06
Metadata version
Version 01 (July 2020)