Green Prosperity: Community-Based Off-Grid Renewable Energy Grant Portfolio 2017, Independent Evaluation

Taken as a whole, this evaluation aims, to the extent possible, to validate the program logic underlying the portfolio of community-based off-grid renewable energy grants in the Green Prosperity Grant Facility, doing so through a focused investigation of two specific grants: W3A Akuo Energy Solar/Micro-Hydro, Berau and W3A Anekatek Solar, East Sumba. It simultaneously aims to measure impacts and compare and contrast how the grants operate, both in terms of how similar programs operate in different contexts within Indonesia and in terms of how programs with different approaches to electrification and community engagement operate. It chiefly investigates changes in energy consumption, energy expenditure, economic opportunity, and greenhouse gas emissions as a result of exposure to a solar photovoltaic (PV) micro-grid that is owned and operated by a community-level Special Purpose Vehicle in treatment villages and sub-villages of the East Sumba and Berau regencies. This report describes baseline conditions of these outcomes of interest and assesses the validity of the project logic in the context of these conditions.To do so it will employ a quasi-experimental methodology in East Sumba combining statistical matching and difference-in-difference analysis techniques as well as non-experimental quantitative and qualitative pre/post performance evaluation methodologies in both East Sumba and Berau.

At baseline, we find the use of solar technology (typically 20 watts or less) more prevalent in East Sumba than in Berau, where individual or shared generators are often used. Although households in East Sumba have access to electricity for more of the day (9.2 vs. 5.3 hours), households in East Sumba consume significantly more diesel and gasoline and have more electric lighting fixtures lit per hour than are lit in East Sumba, such that each regency consumes around 27 lamp-hours per day of electric lighting. Households in Berau spend more on energy absolutely and as a percent of total income, although they are over twice as wealthy on average as households in East Sumba. Our findings lend support to the notion that access to the micro-grids is likely to increase the consumption of energy and decrease expenditure, although the latter outcome is more likely in places like Berau where baseline fuel consumption and expenditure is high than in East Sumba, where it is already low. The validity of this assumption will depend on the increase in electrical capacity from access to the micro-grid relative to other solar sources (between 400 and 900 watts at minimum depending on the regency) allowing households to substitute away from diesel-powered generators, which they currently must use to supplement low-capacity solar home systems.

We find that, with respect to increased economic opportunity, although local enterprises are keen to utilize newly available renewable energy for marginal improvements to their businesses, primarily local economies in each site are likely to preclude any more than marginal changes in economic outcomes of interest such as occupational income, income from transformed agricultural products, and time spent on income generating activity. With respect to greenhouse gas emissions, we find baseline emissions to total 41.00 tons CO2e/month in East Sumba and 44.08 tons CO2e/month in Berau after accounting for diesel, gasoline, and kerosene consumed per month by each household in treatment areas and additional diesel contributed to communities in-kind for community generators from nearby extractive industry corporate social responsibility programs. Similar to energy expenditures, there is more potential for this figure to decrease in Berau, where fuel consumption is high at baseline, than in East Sumba, where fuel consumption per household is already low.

Finally, we assessed the potential for the SPV approach to render the projects sustainable over the long-term. To this end, we find strong community and grantee engagement and capacity for local governance to reflect positively on the potential for sustainability, although there is concern by many stakeholders that the grantees may not remain engaged long enough to transfer the necessary human capacity to villagers for operation and maintenance of the SPV and the micro-grid. Future data collection efforts will observe how the SPV interacts with obstacles and how engaged the respective parties remain.

To assess the validity of our own experimental design in the context of our baseline data, we re-ran our power analysis with our primary data, assessed our treatment and control groups for balance, and conducted a preliminary matching exercise. We find that, barring significant losses from contamination or pruning, we retain sufficient power to detect outcomes of interest. Treatment and comparison groups differ on key covariates, specifically related to wealth and remoteness, but they are equivalent even prior to matching on outcomes of interest such as energy access and electric lighting consumption. Matching techniques are able to eliminate as many as 1/3 of the significant differences detected. Although remaining differences and potential contamination remain risks to our experimental design, these should be mitigated by difference-in-difference analysis techniques and oversampling of control households.