Metolong Program 2019, Independent Impact Evaluation

BACKGROUND: MCC's $362.5 million (2008-2013) Lesotho Compact provided $105.6 million for the Metolong Program's bulk water supply and the Urban and Peri-Urban Water Activity's rehabilitation and expansion works. These investments were based on the theory that improving water supply infrastructure would increase access to and reliability of quality water. This was expected to lead to time savings and reduced illness for households, thereby increasing incomes, and to increased investment by industry. A final performance and impact evaluation was designed to answer questions related to implementation and functionality, management and sustainability, and ultimate impacts on consumers.

METHODS: This ex-post evaluation includes both performance and impact evaluation methodologies and was conducted in two phases. The first phase included a process evaluation with an implementation fidelity exercise, for which data collection included site visits, key informant interviews, document review, and secondary data analysis. In the second phase, a summative evaluation was conducted, which included two quasi-experimental impact evaluation designs using propensity score matching to measure Metolong Program and Urban and Peri-Urban Water program impacts of improved access and supply in intervention areas where implementation fidelity was highest, plus Customer Surveys in all other towns. Water quality tests (for E. coli and free chlorine residual at the tap and E. coli at the point of consumption), were carried out in all towns, and focus groups were carried out in all impact evaluation and customer survey sites as well as in a small selection of rural villages along the Metolong downstream conveyance to investigate unanticipated effects. All surveying was conducted face-to-face using electronic data collection on mobile devices. Other data collection focused on industry is described further in the evaluation report.

FINDINGS: Process Evaluation --The Metolong Program, centered around the construction of a new dam, was implemented faithfully. Highly functioning water treatment works increased bulk water supply to Maseru and its surrounds by 75-94 megaliters per day. The Urban and Peri-Urban Water Activity, centered in the rehabilitation, improvement, and expansion of urban water networks, increased storage capacity and network coverage while installing poorly designed intake and treatment plant infrastructure in some sites, requiring repair at the water utility's expense. Impact Evaluation -- Households in townships supplied by Metolong and those not supplied by Metolong alike reported already high levels of service reliability before the intervention. Accordingly, no significant impacts on time savings, water collection, water consumption, diarrheal illness, or water expenditures were detected for households supplied by Metolong. However, it is likely that given the design of the Metolong Program, its actual impact is to assure the supply in Maseru and surrounds into the long-term, rather than having any immediate acute effects at the household level. In peri-urban Maseru and in Semonkong, newly-connected households experienced significant time savings, increased water consumption, and decreased water collection. Unconnected households also experienced time savings, suggesting some positive spillovers. Households with new connections also reported lower prevalence of diarrheal illness among children under five, but differences are statistically insignificant, partially due to smaller than anticipated sample sizes.