Impact of Teacher Training and Information for Accountability 2008-2010
Liberia, 2008 - 2010
Identification
LBR_2008_EGRA_v01_M
Impact of Teacher Training and Information for Accountability 2008-2010
| Name | Country code |
|---|---|
| Liberia | LBR |
Impact Evaluation Study [ie/ies]
To improve quality of primary education in Liberia after devastating civil war, Liberian Ministry of Education, World Bank and U.S. Agency for International Development (USAID) launched the Early Grade Reading Assessment (EGRA) Plus: Liberia program. The research was an experimental intervention, evaluating how reading assessment in grades 2 and 3, teacher training and community involvement relate with improved education outcomes.
World Bank and USAID funded EGRA Plus: Liberia program. U.S.-based research company RTI International and its local sub-contractor the Liberian Education Trust implemented and evaluated the project.
Baseline, midterm, and final assessments were conducted in 2008-2010 and the results were judged against agreed-upon targets for improved student performance. The baseline test was carried out in November 2008, the midterm assessment was conducted in June 2009, and the final evaluation took place in June 2010.
Preceding this impact evaluation research, the pilot Early Grade Reading Assessment (EGRA) study was done in Liberia in June 2008.
EGRA Plus experiment was applied in 180 schools across Bomi, Bong, Gbapolu, Lofa, Margibi, Monteserrado, and Nimba counties. The sampled schools were randomly assigned to three different groups (60 schools in each group).
The first was a control group that received no program interventions, but whose performance was measured (without alerting there would be repeated measurement). The second, a light intervention group was a set of schools where parents and community members were provided student achievement data in the area of literacy; they were made aware that there would be testing again. Light intervention teachers were trained in the development of a student reading report card, which they issued four times a year. The final, a full intervention group received intensive teacher training, targeting reading instructional strategies. In addition, the full treatment group had the same student achievement information sharing that was provided to parents and communities in light treatment schools.
As part of the June 2010 final assessment, the World Bank through the Spanish Trust Fund for Impact Evaluation (SIEF) sponsored an implementation of the Early Grade Mathematics Assessment (EGMA). The test had two main goals: first, establish a baseline in mathematics that can be used for future efforts, and second, determine if EGRA Plus intervention that was focused on improvements of reading had any secondary impact on improvement of student performance in mathematics.
Overall, 2,988 students were tested during the baseline assessment, 2,882 - during the midterm evaluation and 2,688 - during the final test.
Sample survey data [ssd]
- grade 2 and 3 students;
- teachers;
- head teachers (principals).
Scope
The scope of the study includes:
- Early Grade Reading Assessment (EGRA);
- Early Grade Mathematics Assessment (EGMA);
- classroom observation;
- head teachers' (principals) demographic characteristics;
- teachers' demographic characteristics;
- students' demographic characteristics;
- school characteristics;
- Parent Teacher Association (PTA);
- teacher and school teaching resources;
- teacher training;
- parent involvement;
- students' learning environment;
- students' living standards.
Coverage
Bomi, Bong, Gbapolu, Lofa, Margibi, Monteserrado, and Nimba counties.
Producers and sponsors
| Name |
|---|
| Ministry of Education, Republic of Liberia |
| U.S. Agency for International Development (USAID) |
| World Bank |
| Name | Role |
|---|---|
| RTI International | Implementation and evaluation of EGRA Plus project |
| Liberian Education Trust | Implementation and evaluation of EGRA Plus project |
| Name | Role |
|---|---|
| United States Agency for International Development | Funding of execution and evaluation of EGRA Plus experiment |
| World Bank | Funding of pilot study, funding of EGRA Plus experiment designing, funding of EGMA |
Sampling
The sampling procedure focused only on Liberian public schools. First, 15 districts were selected in a manner proportional to population. Then, a cluster of four schools was selected in each district.
For the selected districts, researches created a distance matrix of all schools i to all other schools j, calculating the length of the hypotenuse between x(i), y(i) and x(j) and y(j). One school was selected at random in a district. This school was considered the centroid of that district's cluster. After that three closest to it schools were selected.
The sampled schools were randomly assigned to control, light treatment and full treatment groups with the goal to have 60 schools in each group.
During November 2008, a national baseline assessment of early grade literacy skills was performed in 176 schools with 2,988 students. The sample size had to be 180 schools; the four missing schools were assessed in January and February 2009. In each school, either 10 or 20 students were assessed, depending on the size of the school and number of teachers.
The June 2009 midterm assessment was conducted in the same EGRA schools. A total of 175 schools and 2,882 students were included in this survey. The June 2010 final assessment was conducted in 175 schools with 2,688 children. As was the case with the baseline and midterm assessment, either 10 or 20 students were assessed, with the target to have at minimum 10 students from grade 2 and 10 students from grade 3, depending on the size of the school. For all three assessments, students were randomly selected from a school register using a systematic sampling procedure implemented by assessors, rather than teachers, in order to prevent teachers from selecting only the best students. The sampling was done from the students in attendance during the day of the test.
The achieved sample at the final assessment was smaller for control and full intervention schools, but near the target for the light intervention schools. The sampling procedures for each assessment were done randomly and independently of each other. No attempt was made to resample children assessed in a previous assessment. Children from the baseline assessment may also have taken part in the midterm assessment, but because children's names were not used, it is impossible to tell with any certainty.
Detailed information about sampling is provided in "Description of Sampling Procedure for EGRA Plus: Liberia" in external resources.
Survey instrument
The following survey instruments are available:
- EGRA Testing Tool and Student Questionnaire for baseline and midterm assessments;
- EGRA and EGMA Testing Tools and Student Questionnaire for final assessment;
- Teacher Questionnaire for baseline, midterm and final assessments;
- Principal Questionnaire for baseline, midterm and final assessments.
- Liberia Early Grade Reading Assessment (EGRA) tool assessed the following set of skills:
- Orientation to print;
- Letter naming fluency;
- Phonemic awareness;
- Familiar word fluency;
- Unfamiliar (or nonsense) word fluency;
- Oral reading (connected text) fluency;
- Reading comprehension;
- Listening comprehension.
In order to prevent "teaching to the test," or memorization, the three assessments (baseline, midterm, final) used different passages and reading comprehension questions.
- Liberia Early Grade Mathematics Assessment (EGMA) tool had the following sections:
- Number identification;
- Quantity discrimination;
- Missing number;
- Addition;
- Subtraction;
- Multiplication;
- Fractions.
Data collection
| Start | End |
|---|---|
| 2008 | 2010 |
| Name |
|---|
| RTI International |
| Liberian Education Trust |
| Ministry of Education, Republic of Liberia |
Midterm EGRA assessors training was organized in May 11-15, 2009, and was facilitated by the Task Coordinator and EGRA Technical Coordinator. The training was also attended by the Liberian Ministry of Education (MOE) EGRA coordination committee. For any application, EGRA teams always train more assessors than needed, in order to ensure that the assessors who are chosen to be deployed are the best possible performers. The total number of trainees was 47, from which the 18 best assessors were selected. The total number of MOE staff trained at this session was 15.
Final assessment EGRA assessors training occurred May 3-7, 2010, and it was facilitated by the Task Coordinator (Medina Korda), EGRA Technical Coordinator (Ollie White), and RTI's Reading Expert (Marcia Davidson). The total number of trainees in Liberia was 45, from which the 28 best assessors were selected. The total number of MOE staff trained at this training was 17, and five of them were deployed.
Formal inter-rater reliability assessments were used for training and selection in all assessment stages.
Data collection for baseline assessment was done in November 2008. 176 schools were assessed, including 60 control, 59 full treatment and 57 light treatment schools, for a total of 2,957 students.
Data collection for midterm assessment started on May 25, 2009, and lasted four weeks. Nine teams were formed, each consisting of two members. They were both tasked with conducting assessments and had the same responsibilities. One of them was chosen to be the team leader and to make sure that all subtasks were completed. Overall, schools were cooperative and open to assessment based on their familiarity with the project. Teams needed to put in extra effort in a few cases, however, where schools were closed due to the rainy season. Together with the EGRA team, the assessors managed to reach their schools. Out of the total 180 schools, 179 EGRA schools were assessed.
A major incident happened in one district. Residents of a community mistook one of the data collection teams for a group of kidnappers. The misunderstanding was based on the color of the vehicle the enumerators were in. Villagers attacked the team. Fortunately, the assessors received only minor injuries. The EGRA team and the Liberian Ministry of Education conducted an inquiry into what happened and a report was provided to USAID.
Data collection for the final assessment began on May 17, 2010, and ended on June 11, 2010. This allowed for four weeks of data collection in 179 schools. There were nine teams, each team consisting of three members to account for the increased work due to Liberia Early Grade Mathematics Assessment (EGMA). In total, 176 schools were assessed. Four schools were not assessed. Two of these were affected by a car accident (a bridge collapsed under one car carrying several enumerators). One control school refused to be assessed because it was being denied the treatment. One light treatment school also refused to participate for the same reason.
Data processing
An EGRA data entry application was developed in June 2008 by Mr. Farwenee Dormu of the Liberian Ministry of Education (MOE), with guidance and support from RTI International. According to Mr. Dormu, the EGRA database was the first database that the MOE had developed since the end of the conflict in Liberia.
Lessons learned from the November 2008 baseline assessment were used to develop a brief manual for data entry. Entry of the baseline EGRA data was completed at the end of January 2009.
For the midterm assessment, RTI International developed a data entry application using Visual Basic that reduced the time for data entry to a third of what would have been needed previously.
For the final assessment, RTI has been working with a Nicaraguan company Centro de Investigación y Acción Educativa Social (CIASES) to develop an efficient and user-friendly data entry system. The EGRA data entry system developed by CIASES offers a low-cost, sustainable solution for minimizing errors.
Data Access
The use of the datasets must be acknowledged using a citation which would include:
- the identification of the Primary Investigator (including country name)
- the full title of the survey and its acronym (when available), and the year(s) of implementation
- the survey reference number
- the source and date of download (for datasets disseminated online)
Example:
Ministry of Education, Republic of Liberia, U.S. Agency for International Development (USAID), World Bank. Impact of Teacher Training and Information for Accountability (EGRA) 2008-2010, Ref. LBR_2008_EGRA_v01_M. Dataset downloaded from [URL] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Contacts
| Name | |
|---|---|
| Nathalie Lahire | nlahire@worldbank.org |
Metadata production
DDI_LBR_2008_EGRA_v01_M
| Name | Affiliation | Role |
|---|---|---|
| Development Data Group | World Bank | Generation of DDI documentation |
2012-03-02
Metadata version
v01 (March, 2012)