Water Sector 2014

Social Impact (SI) has been contracted by the MCC to measure the impact of the Compact activities on economic and social outcomes. This Impact Evaluation (IE) design report lays out how the SI team aims to establish a causal relationship between program interventions and observed changes in household availability and consumption of different sources of water, household income, household expenditure and household health indicators. It also details our strategy for measuring potential impacts on other sectors (agriculture, utility financial performance, and local enterprises) should these occur in parallel to, or instead of, the expected impacts on households. This IE is, to our knowledge, the first attempt to conduct a rigorous IE design of a large infrastructure project in Jordan. It provides a unique opportunity for the MCC, the GOJ, and the broader development community to understand the impact of a large water investment on income and poverty of urban households and others who are affected by it.

The IE will make every possible attempt to measure the impact of the three inter-linked projects separately, in order for MCC to better understand which component(s) of their investment led to specific changes in outcomes. A comparison of the different impacts will further allow for conclusions about the relative cost-effectiveness of each intervention. It must however be noted that because of the complementarities between these different projects, there are very important limitations in the extent to which the IE design will be able to disentangle the separate impacts of the Water Network Project (WNP), the Wastewater Network Project (WWNP), the As-Samra Expansion Project (AEP). Since all three projects are being implemented in Zarqa at the same time, often in overlapping locations, there will be complicated interactions among them which may make it difficult to identify the incremental impacts of each individual intervention. In addition, a significant element contributing to the economic logic of the Compact investments - which we discuss more thoroughly throughout this report - is an assumed water efficiency improvement that would stem from substitution of conventional freshwater currently used in irrigated areas in the Jordan Valley with an expanded supply of treated wastewater collected in Zarqa. The full extent of this assumed substitution effect in fact relies on both the water and wastewater network investments and not on the AEP. It is further assumed that the conventional freshwater saved by this substitution would be made available for higher value uses by municipal and industrial users, thereby improving economic outcomes.

For the purposes of presentation, we have grouped similar and complementary data collection activities into three components, which are described in detail in Section E of this report. This presentation is not intended to imply that any of these three specific components are non-essential; indeed we make the case that all are necessary if the goal is to adequately measure and reduce the risks of misattribution in the overall Compact impact. This point is made following our presentation of the IE logic and a subsequent discussion of the overarching evaluation framework unifying the three components, prior to presenting the details of those components.

Overview of the impact evaluation logic

As emphasized in pre-project feasibility studies and economic analyses of the Compact investments, the economic case for the MCC investments rests on a complex and interrelated set of hypothesized changes. The linkages between the various components and intermediate and final outcomes, respectively, are depicted in the IE logic shown in Figure ES.1. It is important to note that Figure ES.1 does not directly follow the categorization of impacts promulgated in previous descriptions of Compact impacts (e.g. accompanying the MCC's economic rate of return analysis), for the following main reasons:

1. The impacts included in those analyses were admittedly non-exhaustive, due to data limitations in quantifying them. (For example, effects on enterprises and/or on property values were omitted from the analyses - see Section D of this report.)
2. The purposes of the IE logic are a) to trace the relationships between projects, intermediate outputs, and final outcomes, b) to illustrate the overlapping relationships between project activities and desired outcomes, and c) to draw attention to the underlying assumptions.

The IE logic aims to identify the set of final outcomes (and to a lesser extent the intermediate outputs) we intend to measure and track through our IE design. Importantly, the so-called primary substitution effect (the increased use of blended KTR water in irrigation in the place of freshwater) is not and cannot be measured or shown as a single outcome. Rather, the quantification of this possible benefit stems from analysis that integrates several outcomes and outputs - to be carried out at the conclusion of the IE using data we proposed to collect - that flow through the following connections: a) reduced physical losses (WNP) and b) increased wastewater capture (WNP and WWNP); which lead to c) increased wastewater use in agriculture and d) substitution of King Talal Reservoir (KTR) water for King Abdullah Canal (KAC) water in the Jordan Valley; which together e) change per-capita use of utility water and lead to f) end-user time savings; g) consumer cost savings; h) aesthetic and health benefits; and i) are capitalized in land values. Similarly, understanding the net value of the secondary substitution effect, or the increased use of network water in place of tanker and/or vended water, flows through a complex chain that includes (not in order of importance), a) improved water quality at the point of delivery and b) changes in per capita use of utility water (due to the factors listed above as well as these quality improvements) which are embedded in reduced purchase of c) tanker water and d) vended water; both of which should ultimately appear as consumer e) cost and f) time savings, but may also result in reduced sales and/or profits in the water tanker and vended water industries. In addition, the extent of these primary and secondary substitution effects will likely be mediated (positively or negatively) by changes in utility performance, itself a function of the delivery of improved services.