Threshold (IMAGINE) 2011
Niger, 2011
Identification
NER_2011_MCC-TI_v01_M
Threshold (IMAGINE) 2011
| Name | Country code |
|---|---|
| Niger | NER |
Independent Impact Evaluation
This impact evaluation sought to answer three key questions:
- What was the impact of the program on school enrollment and attendance?
- What was the impact of the program on test scores?
- Were the impacts different for girls than for boys?
The evaluation design selected to estimate the impacts of the IMAGINE program was random assignment. The Government of Niger (GON) chose 204 villages to take part in the evaluation based on certain eligibility criteria, such as the number of school-aged girls in the village, access to water within the village, and proximity to a transportation route. Sixty-five schools were randomly selected to receive the IMAGINE program; the remaining 136 served as control villages. Because the villages were randomly assigned treatment status, villages that received the schools (treatment villages) and villages that did not (control villages) did not systematically differ at the outset of the program. Hence, any subsequent differences in outcomes observed between these two groups of villages can be attributed to the program itself and not to other factors. This design, if properly implemented, is methodologically strong and is seen by many as the gold standard of impact evaluation methods.
Overall, IMAGINE had a 4.3 percentage point positive impact on primary school enrollment, no impact on attendance, and no impact on math and French test scores. The program impacts were generally larger for girls than for boys. For girls, the program had an 8 percentage point positive impact on enrollment and a 5.4 percentage point impact on attendance. The program had no impact on girls’ math scores, though there is suggestive evidence it may have had a positive impact of 0.09 standard deviations on girls’ French test scores. No significant impacts were detected for boys’ enrollment, attendance, or test scores. Finally, impacts were larger for younger children (ages 7-10), than for those between the ages of 10 and 12.
Sample survey data [ssd]
Households
Version
Anonymized dataset for public distribution
Coverage
The IMAGINE program was to be implemented in 20 communes within 11 departments located in every region except for Niamey (Figure I.3). Within these communes, 68 villages were to
receive a variety of IMAGINE interventions for promoting girls’ education.
The sample frame comprises households located in 204 villages that met the preestablished criteria for participation identified by the Niger Ministry of Education. Eligibility criteria included the number of school-aged girls in the village, access to water within the village, and proximity to a transportation route. To ensure equitable representation, the program was implemented nationwide, in all regions except urban Niamey. Two departments each were selected for the Maradi, Tahoua, Tillabery, and Zinder regions, while one department each was chosen from Agadez, Diffa, and Dosso. In each department, two communes were selected, except for Agadez and Diffa, where only one commune was chosen. Finally, within each commune, except Ouarafane and Aguié, 10 villages were identified that met the set criteria, and each village was randomly assigned to either a treatment or control village. In the communes of Ouarafane and Aguié, 12 villages were identified and then randomly assigned as either treatment or control. After reviewing eligibility of treatment and control villages following this assignment, it was determined that two communes needed to be dropped from the evaluation (see Chapter III for details), reducing the number of villages in the survey by 20 villages. Three additional treatment villages were not included in the survey because they were preselected by the Government of Niger and were not part of the random selection. Finally, three control villages in the Arlit department, Agadez region, were not surveyed due to civil unrest and were dropped from the study. As a result, the final sample size is comprised of 57 treatment and 121 control villages.
A random selection of 40 households with school-age children (5-12-years-old) were targeted to be surveyed in each village that formed the evaluation sample. Households in the study are defined as groups of people living together in a common physical space, working together under the authority of a person called head of household, and taking their meals together or from the same supply of food.
To develop the village-level household sampling frame, data collectors first conducted a complete census of all the households in the village and identified those with school-age children. Following the census, 40 households in each village were randomly selected to be surveyed. Interviewers conducted the random selection process by writing the name of each head of an eligible household on a piece of paper, placing them in a hat, and drawing 40 names.
In order to collect school data, interviewers used information collected during the household surveys to identify schools regularly attended by children from each village. Interviewers then selected up to three schools within a 10-kilometer radius to be surveyed for each village. Based on the school registers, we were able to match data collected from 197 schools with 10,858 children in the sample. No further sampling was conducted.
Producers and sponsors
| Name | Affiliation |
|---|---|
| Mathematica Policy Research | Independent Evaluator |
| Name |
|---|
| Millennium Challenge Corporation |
Survey instrument
Mathematica developed two questionnaires: one for households and another for schools. The household questionnaire covered household characteristics, demographics, parents’ attitudes towards education, children’s educational outcomes (enrollment and attendance), as well as a French and math assessment. The school questionnaires covered school characteristics and included a school roster to collect information on student enrollment and attendance. School officials were only asked to report enrollment and attendance information for a student if the parents indicated in the household survey that their child attended that school. Both surveys were developed and conducted as paper questionnaires. Full versions of the final questionnaires and assessments are included in Appendix 2.
Data collection
| Start | End |
|---|---|
| 2011 | 2011 |
The data collection team hired 54 interviewers to collect household and school data. They were organized by linguistic groups into 18 teams, with each team led by an experienced field supervisor. The teams were then assigned a cluster of villages and surveys were conducted simultaneously throughout the country.
Access policy
Millennium Challenge Corporation
Millennium Challenge Corporation
http://data.mcc.gov/evaluations/index.php/catalog/96
Cost: None
Contacts
| Name | |
|---|---|
| Monitoring & Evaluation Division | impact-eval@mcc.gov |
Metadata production
DDI_NER_2011_MCC-TI_v01_M
| Name | Role |
|---|---|
| Millennium Challenge Corporation | Review of Metadata |
2014-04
Metadata version
Version 1.0 (April 2014)
Version 2.0 (June 2015). Edited version based on Version 01 (DDI-MCC-NER-IE-EDU-2011-v1) that was done by Millennium Challenge Corporation.