Questionnaires
Mathematica developed two questionnaires: a household questionnaire and a school questionnaire. The household questionnaire asked about household demographics, children's educational outcomes (enrollment and attendance), and parents' perceptions of education. The school survey asked about schools' characteristics and children's attendance and enrollment.
The household questionnaire drew heavily from several existing questionnaires used widely in developing countries, including the Demographic and Health Survey (USAID), the Multiple Indicator Cluster Survey (UNICEF), and the Living Standards Measurement Study (World Bank). Reliance on these questionnaires provided two important benefits. First, given their wide and successful use in developing countries, including Burkina Faso, they enhanced our confidence in the validity and reliability of the questions in the household questionnaire. Second, reliance on the existing questionnaires allows researchers to compare our results with results from the earlier surveys in both Burkina Faso and other countries. Where necessary, we adapted or added survey questions to yield detailed information to answer the research questions. The household survey included the following modules:
- Household characteristics. This module asked for information about the head of household, such as religion, ethnicity, and education; information about the household itself, including GPS coordinates, construction materials, and water source; and intervention-specific information, such as whether any children were attending preschool (Bisongo) or whether any women were participating in literacy training.
- Household listing form. This module asked the respondent to provide a complete list of all children between 5- and 12-years-old residing in the household. Basic information collected on these children included relationship to the head of household, sex, age, and whether the child had attended school at any time during the 2007-2008 school year.
- Education. This module was administered for all children 5- to 12-years-old who attended school at any time during the 2007-2008 school year. Questions addressed access to textbooks; information about the school attended, including specific interventions such as separate latrines, participation in feeding programs, and attendance; and reasons the parents sent the child to school.
- Child labor. This module was administered for all children 5- to 12-years-old, and asked whether the children were engaged in work for persons outside the household (for pay or in-kind) and whether they performed various chores.
- Mathematics assessment. This module was administered to all children 5- to 12-years-old. Children were shown pre-printed cards and asked to identify numbers, count items, indicate which number was the greater of a pair of numbers, and perform simple addition and subtraction.
- French assessment. This module was administered to all children 5- to 12-years-old. Children were shown pre-printed cards and asked to identify letters, read one- and two-syllable words, and identify the correct noun and verb from a list to fill in a blank in a simple sentence. Examples came from grade 1 and 2 Burkina Faso primary education reading texts.
The school questionnaire was based largely on the World Bank's Living Standards Measurement Study School Questionnaire, modified to address Burkina Faso's educational context and answer the evaluation's research questions. The school survey was administered in two waves. The first wave collected information on school characteristics. The second wave, conducted about five months later along with the household survey, collected attendance and enrollment data for children interviewed in the household survey. Accordingly, Mathematica created two school questionnaire forms. The first included detailed characteristics about the school and a roster to collect overall attendance data. The second included only an attendance roster for students enrolled in the study. Together, the school surveys included the following modules:
- School information. This module included general information about the school, such as name, province, department, and type of respondent.
- School characteristics. This module asked the respondent to provide detailed information about the school, including enrollment, type of school (public or private), textbook availability, and whether the school offered health and feeding programs.
- School personnel characteristics. This module asked respondents to provide information about teachers at the school, including number and gender of teachers, teacher training levels, and whether teachers had participated in gender sensitivity training.
- School physical structure. This module asked about the school's physical structure, such as number of classrooms, availability of desks and chairs, school construction materials, water supply, separate latrines, and the presence of a preschool (Bisongo).
- Student attendance roster. This module was split into two versions. The first was administered during the first visit to the school in conjunction with the modules above. The second was administered by itself during the second visit to the school. The first roster collected information about all students enrolled in the school and whether they were in attendance on that day, were in attendance for the previous three days, and were generally in attendance. The second roster collected information only about those students identified in the household survey as enrolled in school. In addition to the information collected on the first roster, the second roster collected - GPS coordinates, the number of days the school was open during the four previous months, and the number of days the student was absent during the same four months.
Both the household and school questionnaires were first written in English and then translated into French. Mathematica and the University of Ouagadougou collaborated on the translations, ensuring that idiomatic expressions or language usage particular to Burkina Faso was appropriately incorporated. However, in reality French is rarely spoken in rural villages. There are currently 68 languages spoken in Burkina Faso, of which several are unwritten or inconsistently written (Gordon 2005). Faced with the prospect of surveying people in many languages, Mathematica decided that the best approach was to hire interviewers fluent in both French and local languages and train them to translate the instrument as they conducted the interview. In Table C.1, we present the native language of respondents to the household survey.
Household Questionnaire Respondent Native Language / Frequency / Percent
French / 178 / 2.1
Mooré / 3,145 / 37.1
Dioula / 33 / 0.4
Fulfudé / 1,782 / 21.1
Gulmachéma / 2,345 / 27.7
Bwamu / 140 / 1.7
Other Language / 844 / 10.0
Total / 8,467 / 100.1
Once the questionnaires were developed, they were tested in a pilot data collection for which we randomly selected 10 villages-5 treatment and 5 comparison-to be surveyed in May and June 2007. Our aim was to survey households and schools in these villages in order to identify potential problems. The pilot called for interviewer training; conduct of a census and random selection in each village; the identification of schools; conduct of the household and school surveys; and data entry, cleaning, and delivery. A Mathematica team traveled with interviewers and observed them in several villages, talked with village residents, and held a debriefing session with interviewers.
The pilot test identified two key problems. First, the household interview was much too long, averaging more than 90 minutes. To reduce respondent burden, we decreased the number of questions to limit the interview to less than one hour. Second, we determined that several questions were difficult for respondents to answer, particularly those about distances, time, and space. For example, respondents struggled to answer questions about distance from the household to the school or the number of hectares farmed. For questions that we thought important for the analysis, we asked the interviewer for an estimate or sought other measures.[Note: Because both the household and school surveys were substantially modified following the pilot data collection, we did not use the pilot data for analysis. During subsequent data collection, however, all 10 villages included in the pilot data collection were revisited and included in the household and school survey.]
For the school survey, we concluded that it was nearly impossible during analysis to link the students on the school roster with children reported by the household survey as enrolled in school. The reason was the lack of a unique identifier such as a government-issued identification number and the fact that many children shared both the same first and last name. The matching procedure was important in that key measures for the evaluation were school enrollment and attendance. Accordingly, we grew concerned that using the household survey alone to measure school enrollment and attendance might lead to misleading results due to social desirability or other biases. As a result, we developed a procedure whereby matching took place while interviewers were in each village. For this procedure, interviewers first completed the household surveys and then populated the school attendance roster with the names of all children identified in the household surveys as enrolled in a local school. They included the child's household ID and household listing number on the roster. We later used these identifiers to link school data to household data. Once in the school, interviewers used the roster to collect attendance and enrollment information only for children on the roster.