Impact Evaluation of Performance-based Incentives for Teachers 2012
Guinea, 2012
Identification
GIN_2012_PITIE-BL_v01_M_v01_A_PUF
Impact Evaluation of Performance-based Incentives for Teachers 2012
Baseline Survey
| Name | Country code |
|---|---|
| Guinea | GIN |
1-2-3 Survey, phase 1 [hh/123-1]
This study aims to assess the impacts of two types of interventions to be implemented by the Guinea Ministry of Pre-University and Civic Education (MEPU-EC) using performance-based rewards for teacher as motivation-enhancing strategies and providing simple guidance on effective classroom management strategies to encourage their use. In the context of the incentive scheme, teacher performance will be measured using an objective and comparable performance indicator.
The evaluation is designed to answer the following research questions:
- What is the effect of performance-based incentives for teachers on teaching practices and behaviors (absenteeism, time on task) and on student learning outcomes?
- To what extent do recognition rewards trigger different (better or worse) modifications in teachers’ behavior and practices compared to those triggered by in-king rewards?
- What is the effect of providing guidance on effective classroom management on teaching practices and student learning outcomes?
- What is the effect of performance-based incentives when teachers are also provided with guidance on effective classroom management practices? Moreover, cost effectiveness of the different treatment arms will be investigated.
The proposed evaluation strategy is a randomized control trial that spans two academic years (2012-2013 and 2013-2014) and targets all grade 3 and 4 teachers of a nationally representative sample of 420 schools. The first year of the impact evaluation focused on assessing the impact of performance-based teacher incentives only and comparing the two types of incentives: in-kind and recognition. The second year includes the additional intervention of delivering guidance on classroom management. Data on schools, teachers, and students is collected through (unannounced) attendance checks, time on task and general classroom observations (carried out in person and using a video), official inspection visits, administration of student’s curriculum-based standardized Math and French tests, teacher surveys and content-knowledge tests, as well student and principal questionnaires. Costing data will be collected through the financial reports of the IDA project and of the government budget (for the second year incentives and guidance intervention) since all expenditures related to the interventions evaluated are paid through these channels.
Baseline data is documented here.
Sample survey data [ssd]
- Teachers
Version
- v01 (Cleaned baseline data as of June 24, 2014).
The baseline data for this IE was collected to provide a pre-intervention overview of the schools, teachers, and students targeted in the context to the performance-based rewards program and check that the randomization successfully balanced the pre-program characteristics of the three groups of schools.
Two main data collection activities were carried to collect the data:
-
Announced visits to the schools to administer the standardized Math and French tests led by the SNCESE ("Service National de Coordination de l'Évaluation du Système Éducatif"), a service of the MEPU-A, as well as a principal questionnaire and a teacher questionnaire.
-
Unannounced inspection visits to schools led by the General Inspection Department ("Inspection General de l'Education", IGE) of the MEPU-A to establish teacher's baseline inspection scores.
Scope
The scope of the study includes:
- performance-based rewards
- school characteristics and environment
- support and monitoring of teachers
- teachers' demographic characteristics
- teachers' work experience and training needs
- pedagogical practices and languages
- absenteeism and events disturbing teaching
- student learning
- students' demographic characteristics and family background
- teachers' performance recognition or punishment
- French test
- math test
Coverage
National
Producers and sponsors
| Name | Affiliation |
|---|---|
| Marie-Helene Cloutier | World Bank |
| Deon Filmer | World Bank |
| Felipe Barrera | Harvard University |
| Name |
|---|
| Strategic Impact Evaluation Fund |
| International Development Association |
Sampling
The sample was designed to be representative, at the national level, of the target grades' teacher population in public French-speaking schools.
The sampling process at the schools took place as followed:
- The population of public Francophone schools was extracted from the in 2011-12 Education Management Information System database and used as the sampling frame;
- Schools were split into the 15 strata defined to capture the school location (8 regions and 2 zones, namely rural-urban).
- Assuming that the number of teachers and students in grade 2 and 3 in in 2011-12 indicates the numbers of teachers to be expected in grades 3 and 4 in 2012-13, the number of grade 2 and 3 teachers/classes for each school was calculated.
- The number of schools to be selected per strata was established using the Markwardt protocol (this represent the average between selecting a proportional and equal number of schools per strata). The selection probability of each individual school was established using the number of targeted teachers in the school and the number of schools in the strata.
- Using a random starting point and the selection probability, 450 individual schools and 75 replacement schools were selected.
This sample was completed by adding the 16 pre-identified schools where the instruments were piloted in the 2011-12 academic year. Therefore, before launching the baseline fieldwork, a sample of 466 schools was targeted. Within each school, all grade 3 and 4 teachers and all of their students were targeted.
Randomization is at the school level but target beneficiaries are the teachers. 420 schools, all grade 3 and 4 teachers, all grade 3 and 4 students.
While in the field at baseline, the teams were unable to locate some of the schools and some of the located schools turned out not to have the targeted grades and thus had to be taken out of the sample. The final sample contains 420 schools. No replacement schools were used. The final sample therefore differs from the targeted sample and national representativeness is uncertain. It is important to note that this does not reduce the internal validity of the IE design since the random assignment of schools to the different experiment arms was carried-out once the realized sample of schools was stabilized.
Sample of teachers and students: once in the school in May 2012, the aim was to administer a standardized test to all grade 2 and 3 students (who will be grade 3 and 4 students in 2012-13, the first year of the intervention). Furthermore, in October 2012, the aim was to survey/inspect all grade 3 and 4 teachers for the 2012-13 academic year. At baseline, a total of 416 principal and 1177 teachers were surveyed; 1214 inspected, and 23183 students participated in the test.
Within each of the targeted teachers’ classes, the objective was that all students should take part in the test. However, once in the field, the teams were faced with larger schools (in terms of the number of students) than expected and thus did not have enough printed copies of the tests to administer it to all students. In about one third of the visited schools, instead of randomly selected students within each of the classroom, only subset of the classrooms (all students) were selected to participate in the test. Furthermore, when class size was too big, a random selection of students within the selected classes was carried out. There is no reason to believe that the selection of students within a class was not random but there is also no certainty that it was. Finally, because of teacher absenteeism and logistics difficulties, the tests were only administered in 353 out of the 420 targeted schools.
Response rates varies from high in the case of inspection and questionnaires to a little lower in the case of test. Balance analysis indicates that these response rates were orthogonal to treatment.
No weight will be used for the analysis and thus no weight variable is included in the datasets.
Survey instrument
The following survey instruments were used: (i) a questionnaire administered to the school’s principal, (ii) a teacher questionnaire administered to targeted teachers (grade 3 and 4), (iii) a Math test (a few parallel booklets), a French test (a few parallel booklets) administered to students in the targeted teachers’ classrooms, and (iv) an inspection bulletin administered to targeted teachers in the context of two lessons, one in French and one in Math.
Below is presented a more detailed description of the various instruments.
Principal questionnaire (May and October 2012)
A. School and principal identification
B. Demographics
C. Education and professional training
D. Work experience and training needs
E. Pedagogical practices and languages
F. School basic characteristics
G. School environment
H. Interaction with colleagues (subordinate, supervisors, etc.)
I. Support and monitoring of teachers
J. Motivation
Teacher questionnaire(May and October 2012)
A. Class and teacher identification
B. Demographics
C. Class characteristics
D. Education and professional training
E. Work experience and training needs
F. Pedagogical practices and languages
G. Interaction with colleagues
H. Motivation
I. Absenteeism and events disturbing teaching
J. Remuneration
K. Perception of key factors influencing student learning
L. Performance recognition or punishment
Student tests (May 2012)
1.A Identification of school, class, and teachers
2.B. School-related student characteristics
1.C. Student environmental and familial backgrounds
2. French test questions
3. Math test questions
Inspection bulletin (October 2012)
I. Class and inspector identification
II. Teacher identification
III. Summary of scores
VI. General material and spatial classroom arrangement
V.1 Lesson 1 – Identification of the lesson
V.2 Lesson 1 – Teaching and learning material preparation
V.3 Lesson 1 – Lesson planning (according to the Competency-based approach)
V.4 Lesson 1 – Delivery of the lesson
V.5 Lesson 1 – Analysis of the own’s performance
VI.1 Lesson 2 – Identification of the lesson
VI.2 Lesson 2 – Teaching and learning material preparation
VI.3 Lesson 2 – Lesson planning (according to the Competency-based approach)
VI.4 Lesson 2 – Delivery of the lesson
VI.5 Lesson 2 – Analysis of the own’s performance
Data collection
| Start | End | Cycle |
|---|---|---|
| 2012-05-01 | 2012-11-01 | Baseline |
All data collection activities were carried under close supervision of the Impact Evaluation World Bank team which oversaw the training sessions and carried-out random spot check in the field during data collection.
Decentralized government staff (inspectors, district and prefectural education officers, planners, etc.) served as survey administrators. Individuals were selected on a competitive basis (using a test) and trained as enumerators and supervisors. To enhance objectivity, inspectors were assigned to a region different from their regular assignment.
Data Access
The use of the datasets must be acknowledged using a citation which would include:
- the identification of the Primary Investigator (including country name);
- the full title of the survey and its acronym (when available), and the year(s) of implementation;
- the survey reference number;
- the source and date of download (for datasets disseminated online).
Example:
Marie-Helene Cloutier, World Bank; Deon Filmer, World Bank; Felipe Barrera, Harvard University. Guinea Impact Evaluation of Performance-based Incentives for Teachers 2012, Baseline Survey (PITIEL-BL). Ref. GIN_2012_PITIE-BL_v01_M_v01_A_PUF. Dataset downloaded from [URL] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Contacts
| Name | Affiliation | |
|---|---|---|
| Marie-Helene Cloutier | World Bank | mcloutier@worldbank.org |
Metadata production
DDI_GIN_2012_PITIE-BL_v01_M_v01_A_PUF_WB
| Name | Affiliation | Role |
|---|---|---|
| Development Data Group | World Bank | Study documentation |
| Education - GP | World Bank | Study documentation |
2014-09-17