{"type":"survey","doc_desc":{"title":"metadata-lso-rural-2010-v1","idno":"DDI_LSO_2010_MCC-RWSS_v01_M","producers":[{"name":"Millennium Challenge Corporation","abbreviation":"MCC","affiliation":"","role":"Review of Metadata"}],"prod_date":"2015-02-24","version_statement":{"version":"Version 1.0 (February 2015)\nVersion 2.0 (April 2015). Edited version based on Version 01 (DDI-MCC-LES-IE-WASH-2010-v01) that was done by Millennium Challenge Corporation.","version_notes":"The Lesotho Rural Water and Sanitation Project (WSP) in rural areas provided for improved water and sanitation services for 27,245 households or approximately 160,000 persons through construction of new water sys\u00adtems and ventilated improved pit (VIP) latrines. These households are located in 250 villages that were identified by the Department of Rural Water Supply (DRWS) as lacking access to safe drinking water and adequate sanitation."}},"study_desc":{"title_statement":{"idno":"LSO_2010_MCC-RWSS_v01_M","title":"Rural Water and Sanitation Supply 2010-2013","alt_title":"MCC-RWSS 2010-13"},"authoring_entity":[{"name":"NORC at the University of Chicago","affiliation":""}],"production_statement":{"funding_agencies":[{"name":"Millennium Challenge Corporation","abbreviation":"MCC","role":""}]},"distribution_statement":{"contact":[{"name":"Monitoring & Evaluation Division","affiliation":"Millennium Challenge Corporation","email":"impact-eval@mcc.gov","uri":""}]},"series_statement":{"series_name":"Independent Impact Evaluation"},"study_info":{"keywords":[{"keyword":"Lesotho","uri":"","vocab":""},{"keyword":"Rural water","uri":"","vocab":""},{"keyword":"Impact evaluation","uri":"","vocab":""},{"keyword":"Water","uri":"","vocab":""},{"keyword":"Sanitation","uri":"","vocab":""},{"keyword":"Randomization","uri":"","vocab":""},{"keyword":"RCT","uri":"","vocab":""}],"abstract":"The impact evaluation design of the rural water intervention proposes two separate approaches so as to ensure a \u201cdefensive evaluation\u201d that is, so that results can be obtained in spite of unfore\u00adseen events or mishaps. One is a phased randomized control trial (RCT) in which 100 water sys\u00adtems were allocated (roughly) equally to treatment (i.e., early treatment) and control (i.e., late treat\u00ad\u00adment) groups within each of the ten districts of Leso\u00adtho. The other is a continuous treat\u00adment-variable (CTV) model in which the intensity of treatment can be estimated without the need to isolate a separate control group. As we shall see, while our revised design provides tech\u00adniques to overcome the threats to internal validity that delayed implementation created for the RCT design, there are compelling reasons to give a greater role to the CTV approach as well as to carry out end\u00adline surveys.\n\nThe original evaluation design, developed under NORC's first contract with MCC, focused on a randomized design, under which NORC, with MCA planned for a 6-9 month gap between the end of construction and rehabilitation of treat\u00adment water projects (in 50 Phase-A villages) and the start of water pro\u00adjects in the control areas (in 50 Phase-C villages) to ensure sufficient time for impact to be realized in all outcomes of interest. A nine month gap would allow us to mea\u00adsure long-term outcomes such as changes in productivity and income, while a six-month gap would limit the impact analysis to short-term and intermediate outcomes. Under the original design, all Phase-A villages had a largely similar construction timeline with concurrent start and end dates of construction; thus, it was reasonable to expect that there would be a nine-month (or, at a minimum, a six-month) lag between the end of construction of the 50 Phase-A villages and the start of construction of the 50 Phase-C villages.\n\nDelays in the construction of Phase-A (early treatment) water systems, resulted in 11 treatment villages (Phase-A1 villages) under\u00adgoing construction concurrently with the Phase-C control villages. This over\u00adlap has called into question the validity of the original evaluation design and the effi\u00adcacy of admin\u00adistering an end\u00adline. As Table 1 in the Evaluation Design Report demonstrates, construction was completed in only 70 per\u00adcent of Phase-A vil\u00adlages (34 of the 50) 9 months before construction commenced in Phase-C con\u00adtrol villages. For these 34 villages - which belong to the Phase-Arev group - the time of exposure to treatment before controls began receiving treatment between in January 2013 range from 10 to 20 months. Since the mid\u00adline data collection preceded the start of construction in Phase-C villages (i.e., Novem\u00adber to December, 2012), duration of exposure to treatment by the time of midline data collection for these 34 Phase-Arev villages was about 9-19 months. Five villages in Phase Arev and all 11 Phase-A1 villages had not been exposed to treatment by the time of midline data col\u00adlec\u00adtion. The reduction of the treatment sample at midline by 30 percent has implications for the sta\u00adtistical power of the evaluation design to detect impacts at the level gen\u00aderally desired by MCC.\n\nThe Evaluation Design Report discusses the implications of construction delays for the evaluation design, indicates what evaluation questions can and cannot be addressed without an endline, and discusses the pros and cons of alternative design stra\u00adte\u00adgies to deal with the unforeseen complications in implementation.","coll_dates":[{"start":"2010-12","end":"2011","cycle":"Baseline"},{"start":"2012-11","end":"2012-12","cycle":"Midline"},{"start":"2013-04","end":"2013","cycle":"Midline Update"},{"start":"2013-11","end":"2013-12","cycle":"Endline"}],"nation":[{"name":"Lesotho","abbreviation":"LSO"}],"geog_coverage":"The project was implemented in all 10 districts of Lesotho. On average, 25 villages within each of these districts were selected for construction of new rural water systems and accompanying hygiene and training activities.","analysis_unit":"Individual, household","data_kind":"Sample survey data [ssd]"},"method":{"data_collection":{"data_collectors":[{"name":"Bureau of Labor Statistics","abbreviation":"","affiliation":"Government of Lesotho"}],"sampling_procedure":"The sampling frame for the IEMS consists of all villages in Lesotho based on publicly available geospatial data and 2006 Census data. Information on administrative location, geo-coordinates, rural-versus urban designation and population was merged with publicly available physiographic and geographic data to be used as covariates in the sampling. From this central dataset, indi\u00advi\u00addual sample frames were designed and PSUs were selected for two central project components: rural water and urban\/peri-urban water. For rural water, villages were the primary sampling units. \n\nThe centralized frame dataset consists of the following variables:\n\n\u00b7  Primary Sampling Unit identifying information: Village name, Village ID (both GIS ID and Census ID), Village Geo-coordinates (X and Y), Enumeration Area ID, Community Council, District, Constituency\n\u00b7  General covariates: Population, average annual temperature, precipitation, vegetation produc\u00adtivity potential, number of households, urban\/rural designation\n\nSampling Design\nThe random sample selection was sequentially sampled without replacement in the form of a multistage cluster design as follows:\n\nThere are two stages to the design for the rural water intervention. They cover the designation and selection of villages (PSUs, clus\u00adters) and households (SSUs): \n\n\u00b7  Village sample. As described in the Evaluation Design Report, of the 250 villages in 10 districts selected by DRWS for the MCA rural water interventions, 100 villages (10 per district) were deemed \"ready\" for the intervention in 2008. Fifty of these 100 villages were randomly assigned to treatment (Phase-A), while the remaining 50 were assigned to the control group (Phase-C). Final implementation lists, however, only consisted of a random sample of 50 treatment villages, 48 control vil\u00adlages. The village locations are shown on the map in Annex B in the Evaluation Design Report. The only difference between the treatment and control villages was that the control villages would receive treatment after a delay during the evaluation period according to the pipeline design.\n\nUnfortunately, infrastructure construction ran into delays in villages selec\u00adted for treatment in three districts (Mohkohtlong, Qacha's Nek, and Thaba Tseka). These districts are the most remote (and mountainous), hence, likely to be systema\u00adtically different from the other treated vil\u00adlages. The emergence of this group receiving later treat\u00adment required our evaluation design to be retrospectively divided into two treatment groups, T1 (earlier treatment) and T2 (later treat\u00adment).\n\n\u00b7  Household sample. Within each treatment and control village a systematic ran\u00addom sam\u00adple of 13 households was selected. The interview was conducted with the head of the household or the person in the house\u00adhold most knowledgeable about household water and sanitation issues.","research_instrument":"The main data sources for this evaluation are the Impact Evaluation Multipurpose Survey (IEMS) and the suite of Activity Monitoring Plan (AMP) surveys.\n\nThe IEMS is a longitudinal analytic survey specifically designed to collect data for the impact evaluations of the Lesotho Compact health and water activities. Three rounds of the IEMS were originally proposed in the evaluation design: a baseline, midline, and endline. These three rounds were approved by MCC. To date, the Bureau of Labor Statistics (BoS), under an Implementing Entity Agreement (IEA) with MCA-Lesotho, has conducted the baseline and midline IEMS. \n\nThe AMP surveys are a suite of checklists and data collection forms developed by Cowater, in conjunction with DRWS and MCA, to monitor the construction and training activities associated with the Lesotho WSP. Of these, NORC has refined and expanded three surveys to collect data needed for the impact evaluation. These surveys would be adminis\u00adtered to community (household) members, VWHC members and Water Minders in rural water villages along with the endline IEMS. The sample for these surveys would overlap with the vil\u00adlages and households in the IEMS.\n\nThe purpose of the surveys, which the evaluator proposes to implement with the endline IEMS, are described below:\n\u00b7  Hygiene Awareness & Latrine and Environmental Hygiene (HALEH). Assesses the house\u00adhold's knowledge and practices of proper hygiene and sanitation. The questionnaire consists of two parts: PHAST awareness and observation of the VIP latrine and the household's envi\u00adronmental surroundings. The HALEH sample will consist of 13 interviews per village, admin\u00adistered to house\u00ad\u00ad\u00adholds that were interviewed during the IEMS baseline and midline. For efficiency HALEH will become an additional module of the IEMS question\u00adnaire at endline. The HALEH module will provide an endline assessment on the link between the level of hygiene training received by the beneficiary and hygiene and sanitation know\u00adledge and practices.\n\n\u00b7  Village Water and Health Committee Functionality (VWHC). Assesses the degree to which the members of the VWHC are able to pro\u00advide monthly rou\u00adtine operational man\u00adage\u00adment to the village's water system. It also asses\u00adses the current sta\u00adtus of the system's oper\u00ada\u00adtion. The VWHC ques\u00adtionnaire will be administered to the committee in a group setting at endline.\n\n\u00b7  Water Minder Expertise (WM). Assesses the level of training that the WM received by determining his level of knowledge in after care main\u00adtenance and in repairs of the sys\u00adtem. It will be administered to each Water Minder in the village (approximately 2-3) at endline."}},"data_access":{"dataset_availability":{"access_place":"Millennium Challenge Corporation","access_place_uri":"http:\/\/data.mcc.gov\/evaluations\/index.php\/catalog\/137","original_archive":"Millennium Challenge Corporation\nhttp:\/\/data.mcc.gov\/evaluations\/index.php\/catalog\/137\nCost: None"},"dataset_use":{"conf_dec":[{"txt":"","required":"no","form_no":"","uri":""}]}}},"data_files":[],"variables":[],"variable_groups":[]}