Improving General Education Quality 2015-2017, Independent Impact Evaluation
Georgia, 2015 - 2017
Get MicrodataIdentification
GEO_2015-2017_MCC-IGEQ_v01_M
Improving General Education Quality 2015-2017
Independent Impact Evaluation
| Name | Country code |
|---|---|
| Georgia | GEO |
The school rehabilitation activity seeks to decrease student and teacher absenteeism, increase students’ time on task, and, ultimately, improve learning and labor market outcomes. We propose a mixed-methods study design, with three components: (1) a process evaluation examining the program’s implementation and costs; (2) a randomized controlled trial (RCT) impact evaluation using a school-level stratified random assignment design, and (3) in-depth analysis of the relationship between changes in school infrastructure and changes in the learning environment, using qualitative methods in a subset of study schools.
Sample survey data [ssd]
Individuals
Scope
| Topic | Vocabulary |
|---|---|
| Education | MCC Sector |
Coverage
The exact number of rehabilitated schools is still being determined but is expected to include up to 130 schools throughout Georgia. The study sample will include a control group of schools that is approximately equal in size to the number to the treatment schools.
To estimate the impacts of the school rehabilitation activity, our study uses a school-level, stratified random assignment design. Schools assigned to the treatment group will at minimum receive detailed rehabilitation design assessments, and-where rehabilitation is feasible-treatment schools will receive the program's full set of infrastructure rehabilitation services.
Producers and sponsors
| Name |
|---|
| Mathematica Policy Research |
| Name |
|---|
| Millennium Challenge Corporation |
Sampling
To develop the random assignment procedure, the design first stratifies the sample of schools by region. Within regions, we then consider the benefits of further stratifying the sample on the following school-level characteristics:
· Total enrollment
· Secondary enrollment (students in grades 10 to 12)
· Size of school building
· Government rating of school infrastructure conditions
· Minority language status (indicator for instruction primarily in Azeri or Armenian)
· Rural status (indicator for school located in a village or mountainous area)
· Average baseline test scores in math, history, and literacy
During the 2013-2014 school year (before random assignment), MCA-G hired a design contractor and partially or fully completed rehabilitation designs for several schools in program regions. Due to implementation delays, no rehabilitation work took place in these schools during the 2014 summer construction season, meaning the predesigned cases could be included in the random assignment pool for this evaluation. In total, 29 program-eligible schools have existing rehabilitation designs. To realize cost savings from this prior design work, at the request of MCA-G and MCC, the evaluation will give the predesigned schools a higher probability of being assigned to treatment (66 percent) than the schools currently lacking designs. To do so, our approach places the pool of predesigned schools in its own separate set of random assignment blocks. The study’s impact analyses will adjust statistically for differences in the probability of selection into treatment associated with these predesigned strata.
Survey instrument
Student survey: Survey data on student characteristics, recallbased measures of attendance (to be validated using site visits), perceived determinants of student attendance, perceptions of school building quality and safety, self-reported respiratory health, and perceptions of time on task during the school day. This will also include an assessment of student learning.
Parent Survey: Survey data on family demographics and socioeconomic characteristics, recall-based measures of student attendance, perceived determinants of student attendance, and perceptions of school building quality and safety.
Teacher Survey: Survey data on teacher experience, demographic characteristics, certifications, perceptions of the quality and safety of school facilities, recall-based measures of time spent on instruction, and student attendance records.
School Director Survey: Survey data on school director operations and maintenance practices, average operations and maintenance expenditures, school facility usage, and student attendance records.
School Building Infrastructure Assessments
Data collection
| Start | End | Cycle |
|---|---|---|
| 2015-04-30 | 2015-06-07 | Baseline data collection in Phase I regions (Mtskheta-Mtianeti, Racha- Lechkhumi and Kvemo Svaneti, Samtskhe-Javakheti, and Shida Kartli) |
| 2016-04-01 | 2016-05-29 | Baseline data collection in Phase II regions (Kakheti and Kvemo Kartli) |
| 2017-02-06 | 2017-04-17 | Baseline data collection in Phase III regions |
| 2017-02-06 | 2017-02-28 | One-year follow-up for Phase I regions |
| 2017-02-05 | 2017-02-27 | Two-year follow-up in Phase I regions |
| Name | Affiliation |
|---|---|
| Institute for Polling and Marketing | |
| National Assessment and Examination Center | Georgia |
Due to the staggered approach to data collection, the 2017 data collection round will be the only year in which the local data collection firm conducts site visits in all of the evaluation’s treatment and comparison schools. Thus, in most years the number of visited schools will be smaller, potentially reducing the logistical burdens associated with the data collection effort.
Access policy
Millennium Challenge Corporation
https://data.mcc.gov/evaluations/index.php/catalog/151
Cost: None
Data Access
| Is signing of a confidentiality declaration required? |
|---|
| No |
Mathematica Policy Research. Evaluation Design Report – IGEQ School Rehabilitation Activity. March 2015.
Contacts
| Name | |
|---|---|
| Monitoring & Evaluation Division of the Millennium Challenge Corporation | impact-eval@mcc.gov |
Metadata production
DDI_GEO_2015-2017_MCC-IGEQ_v01_M
| Name | Role |
|---|---|
| Millennium Challenge Corporation | Metadata Producer |
2015-04-16
Metadata version
Version 1.0 (April 2015)
Version 2 (June 2020). Edited version based on Version 1 (DDI-MCC-GEO2-MPR-IGEQ-2015-v1) that was done by the Millennium Challenge Corporation.
The evaluation will focus on estimating the impacts of school rehabilitation on students and teachers. In particular, the evaluation’s findings will pertain to the population of students enrolled or potentially enrolled in the types of schools selected to receive rehabilitation services. The evaluation’s primary sample of interest will be the population of students enrolled in grade 8 or grade 10 at baseline in study schools—we will track these two cohorts of students over time and measure their outcomes during two follow-up years.