Impact Evaluation of PNPM Generasi Program 2009-2010
Indonesia, 2009 - 2010
Get MicrodataIdentification
IDN_2009_PNPM_v01_M_v01_A_PUF
Impact Evaluation of PNPM Generasi Program 2009-2010
Third Wave
| Name | Country code |
|---|---|
| Indonesia | IDN |
Integrated Survey (non-LSMS) [hh/is]
The PNPM Generasi Impact Evaluation project was carried out in three waves. The baseline survey took place from June 2007 to August 2007. The second wave was conducted from October 2008 to January 2009, after 15 to 18 months of Generasi implementation. The third and final evaluation survey was implemented from October 2009 to January 2010 after 27 to 30 months of project implementation.
Over 45,000 household members, village heads, and school and health facility staff were respondents for the third and final round of survey.
The Generasi project focuses on the following indicators of maternal and child health and educational behavior:
Health Indicators
- Four prenatal care visits
- Taking iron tablets during pregnancy
- Delivery assisted by a trained professional
- Two postnatal care visits
- Complete childhood immunizations
- Adequate monthly weight increases for infants
- Monthly weighing for children under three and biannually for children under five
- Vitamin A twice a year for children under five
Education Indicators
9. Primary school enrollment of children 6-to-12 years old
10. Minimum attendance rate of 85 percent for primary school-aged children
11. Junior secondary school enrollment of children 13-to-15 years old
12. Minimum attendance rate of 85 percent for junior secondary school-aged children.
PNPM Generasi known as the National Community Empowerment Program - Healthy and Smart Generation (Program National Pemberdayaan Masyarakat-Generasi Sehat dan Cerdas) began in mid-2007 in rural areas of five Indonesian provinces selected by the government: West Java, East Java, North Sulawesi, Gorontalo, and Nusa Tenggara Timur. The Generasi project is implemented by Indonesia’s Ministry of Home Affairs, and is funded through government of Indonesia resources, loans from the World Bank and grants from several bilateral donors.
In Generasi, all participating villages receive a block grant each year to improve education, and maternal and child health in their villages. The village block grants ranged from an average of $8,500 in 2007 up to an average of $18,200 in 2009.
In order to evaluate the overall impact of Generasi, as well as to separately identify the impact of Generasi's performance incentives, program locations were selected by lottery to form a randomized, controlled field experiment. Each location was randomly allocated to one of two versions of the program: an "incentivized" treatment with the pay-for-performance component (treatment A) and an otherwise identical "non-incentivized" treatment without the pay-for-performance incentives (treatment B).
The impact evaluation project was conducted from 2007 until 2010, in three waves. Documented here is the third round, carried out from October 2009 to January 2010.
The sample for the research covered each of the 300 subdistricts that were included in the original Generasi randomization. In each subdistrict, eight villages were randomly selected (unless the subdistrict had fewer than eight villages, in which case all were selected). Overall, 2,313 villages were sampled in each of the three survey waves.
The data for the impact analysis was gathered from surveys of households, mothers, health service providers, and school and village officials.
Sample survey data [ssd]
- Households,
- Married women, age 16-49,
- Children age 6-15, respondent: mother of the child,
- Children age < 3, respondent: mother of the child,
- Villages,
- Community health centers (Puskesmas),
- Village midwives,
- Village health posts,
- Junior secondary schools.
Version
v01,
Edited, anonymized datasets
Scope
Households: Household roster, deaths in previous 12 months, migration, water/sanitation, receipt of government poverty programs, participation in non-formal education, consumption, assets, economic shocks, health insurance, morbidity, outpatient care use, social capital, knowledge and participation in PNPM/KDP activities.
Married women age 16-49: Fertility history, use of health services during pregnancy, inspection of Generasi coupons (Wave II), family planning, health and education knowledge.
Children age 6-15: School enrollment, attendance, grade repetition, cost of schooling, scholarships, child labor.
Children age < 3: Growth monitoring (posyandu), immunization records, inspection of the Generasi coupons (Wave II), motor development (Wave III), breastfeeding and nutritional intake, weight measurement, height measurement (Waves I & III).
Home-based tests: Test of math and reading skills administered at home (separate test for ages 6-12 and ages 13-15) (Waves I & III)
Village characteristics: Demography of the village, hamlet information, access to health services and schools, economic shocks, access to media, community participation, daily laborer wage rate, development projects in the village (Waves II & III).
Community health centers (Puskesmas): Head of facility background, coverage area, budget, staff roster, time allocation of head doctor and midwife coordinator, service hours, services provided, fee schedule, number of patients per service during the previous month, medical and vaccine stock, data on village health post, participation in Generasi (Waves II & III), direct observation regarding cleanliness.
Village midwives: Personal background, location of duty, condition of facility, time allocation, income, services provided, fee schedule (public and private), experiences during past
three deliveries, number of patients seen per service during the previous month, equipment and tools, medical supplies and stock, village health post management, participation in Generasi (Waves II & III), structure of subsidies received.
Primary school (Waves II & III), Junior secondary school: Principal background, principal time allocation, teacher roster, school facilities, teaching hours, enrollment records, attendance records, offi cial test scores, scholarships, fees, budget, participation in Generasi (Waves II&III), direct observation of classrooms, including random check on classroom attendance.
Village health post cadre (Waves II & III): Respondent characteristics, health post characteristics, service providers, cadre roster, tools and equipment, participation in Generasi.
Coverage
West Java, East Java, North Sulawesi, Gorontalo, and Nusa Tenggara Timur provinces
Producers and sponsors
| Name | Affiliation |
|---|---|
| Susan Wong | World Bank |
| Junko Onishi | World Bank |
| Ben Olken | MIT/JPAL |
| Ministry of Home Affairs | Government of Indonesia |
| Name |
|---|
| Spanish Impact Evaluation Fund, World Bank |
| Government of Indonesia |
Sampling
In order to evaluate the overall impact of Generasi, as well as to separately identify the impact of Generasi's performance incentives, Generasi locations were selected by lottery to form a randomized, controlled field experiment.
The Generasi randomization was conducted at the subdistrict (kecamatan) level, so that all villages within the subdistrict either received the same treatment of Generasi (treatment A or treatment B) or were in the control group. Randomizing at the subdistrict level is important since many health and education services, such as community health centers (puskesmas) and junior secondary schools, provide services to multiple villages within a subdistrict.
The Generasi locations were selected through the following procedure. First, 300 target subdistricts were identified, targeting poor, rural areas that had an existing community-driven development infrastructure.
Each subdistrict was then randomly assigned by computer into one of three equal-sized groups: treatment A, incentivized (100 subdistricts); treatment B, non-incentivized (100 subdistricts); or control (100 subdistricts). Within a subdistrict, all villages received the same treatment. The randomization was stratified by district (kabupaten), to ensure a balanced randomization across the 20 different districts in the study.
The sampling design for the household component of the Generasi surveys was chosen to ensure adequate coverage in the key Generasi demographic groups: mothers who recently were pregnant or gave birth, children under age three, and children of school age. Within each village, one hamlet (dusun) was randomly selected, and a list of all households was obtained from the head of the hamlet. Five households were randomly sampled from that list to be interviewed. These households were stratified so that two selected households had at least one child under age two, two selected households had a child under age 15 but no children under age two, and one household had no children under age 15.
Data collection
| Start | End |
|---|---|
| 2009-10 | 2010-01 |
| Name |
|---|
| University of Gadjah Mada |
Data Access
Use of the dataset must be acknowledged using a citation which would include:
- the identification of the primary investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download.
Example:
Susan Wong, World Bank; Junko Onishi, World Bank; Ben Olken, Massachusetts Institute of Technology; Ministry of Home Affairs, Government of Indonesia. Indonesia - Impact Evaluation of PNPM Generasi Program (PNPM) 2009-2010, Third Wave. Ref. IDN_2009_PNPM_v01_M_v01_A_PUF. Dataset downloaded from [URL] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Contacts
| Name | Affiliation | |
|---|---|---|
| Susan Wong | World Bank | Swong1@worldbank.org |
Metadata production
DDI_IDN_2009_PNPM_v01_M_v01_A_PUF
| Name | Affiliation | Role |
|---|---|---|
| Antonina Redko | World Bank | Generation of DDI documentation |
2012-07-25
Metadata version
v01
First version of metadata documentation (July 2012)