Health Care Centers and ART Clinic Infrastructure 2010, Independent Evaluation
Lesotho, 2010 - 2011
Get MicrodataIdentification
LSO_2010_MCC-HCCACI_v01_M
Health Care Centers and ART Clinic Infrastructure 2010
Independent Evaluation
| Name | Country code |
|---|---|
| Lesotho | LSO |
The Health Center activity, in its conception, focused on the rehabilitation of up to 138 health centers via the provision of power, water, telecommunications equipment, medical equipment, instruments and clinical furniture. The activity also included a services component comprised of nurse training, public health management, and waste management.
Originally half of the health centers were assigned to treatment and half to the control. However, due to other donors' activities, construction sequencing and budgetary considerations, only 59 of the 138 HC were included in the impact evaluation. Thirty-one health centers, including three high need health centers, were assigned to the treatment group, and 28 health centers were assigned to the control group. All but the 3 high need health centers were assigned randomly to each group. To enhance precision, treatment and control health centers were then clustered by need and eco-region and matched to one another. The evaluation used a randomized rollout design whereby the treatment group would receive the intervention early and the control group would receive the intervention later.
The ART Clinics activity aimed to improve infrastructure in outpatient departments (OPDs) in 14 hospitals through rehabilitation and/or construction, provision of power, water, and telecommunications equipment, and medical equipment and instruments and clinical furniture.
Since randomization was not feasible for this activity, the evaluation design originally called for a two-way evaluation that used both difference-in-difference and before-and-after estimation techniques. This approach would have allowed NORC to estimate outcomes and, in a subset of indicators, cross-validate results, in order to have the most reliable possible assessment of the effect of the OPD intervention. This evaluation design assumed that 5 OPDs, the minimum number required to provide a quasi-experimental design with acceptable statistical power, would serve as controls. However, there are only 16 OPDs in Lesotho, which meant that only 2 OPDs will serve as controls. Hence, the final proposal was that NORC use only the before-and-after estimator, supported by qualitative data to fill in gaps and interpret unexpected findings.
Ultimately, the randomized rollout design that was developed for the Lesotho Health Centers was rendered infeasible due to infrastructure delays. Before pursuing an alternative methodology, MCC and the evaluator revisited the underlying logic and targeted outcomes for the Health Project and determined that compact-supported infrastructure, which was the focus of the exisiting evaluation contract and designs for Health Centers and OPDs, could not be evaluated in isolation. MCC opted to cancel that contract and pursue a more comprehensive approach to evaluating the entire Health Project.
Four separate sets of data were collected before the original evaluation was canceled: two short surveys (a survey of facility staff and an exit interview of patients) and two types of administrative data (register of patients undergoing anti-retroviral therapy and monthly summary of all outpatients).
Sample survey data [ssd]
The unit of analysis is ultimately the health facility (clinic or hospital outpatient department); this is also the unit of observation in the outpatient monthly summary dataset. However, in the exit survey and ART register, each observation represents an individual patient, and in the staff survey, each observation is one staff member.
Version
Anonymized dataset for public distribution
Scope
| Topic | Vocabulary |
|---|---|
| Health | MCC Sector |
| Gender | MCC Sector |
Coverage
National coverage - all ten districts of Lesotho are represented.
The sampling frame for the patient exit and staff survey consists of 59 Health Center Facilities that were selected as part of a randomized treatment/control experimental design and 16 Out-Patient Departments within hospitals across Lesotho. However, three of the 75 facilities were ultimately not covered by any of the four datasets.
Producers and sponsors
| Name |
|---|
| National Opinion Research Center at the University of Chicago |
| Name |
|---|
| Millenium Challenge Corporation |
Sampling
Methodology
a. Frame
For the health center and OPD data collection, the sample consisted of 75 facilities - 59 health clinics (31 treatment, 28 control) and 16 OPDs (14 treatment, 2 control).
The evaluation included several datasets; the sampling procedure for each follows:
1: Exit Patient Survey: A systematic random sample was used to select 26 patients per day over the course of two days at each Health Center or OPD. The sampling protocol was as follows:
A. Identify the average number of patients per day numerical range for the facility.
B. Let N be the average number of patients seen per day by a facility.
C. Set the sample size n at 13.
D. Calculate the step size k=N/n and Round Down to the nearest integer k.
E. k was the systematic sample interval for the selection of exiting patients at the facility.
F. k* was used as the systematic sample interval during each day of exit interviews over the course of a two day visit at a health facility.
Example: Maseru Defense Force has on average a population of a 100 visiting patients per day.
N=100, is the average population size. Sample size n is set to 13. [N/n] = [100/13] ˜ 7.
Therefore, step size k*=7 is the systematic sample interval for the selection of exiting patients to be inclusive in the sample for exit interviewing at Maseru Defense Force Center.
2: Staff Questionnaire: The Staff Questionnaire was administered to doctors, nurses, nurse assistants, and counselors at Health Centers and OPD as a self-administered anonymous survey. It was administered to staff during arrival, and collected at the end of the health center's day by a survey team member. Observation visits to Health Centers reported that at minimum a Health Center will have 3 staff and at maximum 12. Outpatient departments at hospitals can have more than 15 of these staff in total. At maximum a total number of 15 questionnaires could be completed by the staff at health facilities.
3: Out-patient Monthly Summary Report Population Totals & HIV/AIDs New Enrollments: The Administrative data was recorded into an Excel spreadsheet and documented exactly as recorded in the Health Center's file.
4: ART Cohort Study: A maximum number of 30 ART patient records was collected daily during Health Center operation hours. The initial sample size was set to achieve a maximum number of 25 complete patient records during systematic sampling data collection, given that some ART patient records may be incomplete due to attrition, transfers, or other unknown factors for which there is no required documentation. To account for incomplete records, ART patient records were oversampled by 20% to achieve a total of 30 patient records during data collection. ART registers from months February 2010 to July 2010 at a Health Center, a 6 month time period, comprised the sampling frame from which the ART patient records were selected for inclusion in the sample.
The sampling protocol for the selection of 30 ART patient records at a Health Center by a NORC staff member follows:
A. Identify the ART register(s) containing the targeted 6 months (February 2010 to July 2010)
B. Count the number of pages containing these records (Let P=number of pages)
Note: Each page in the register contains 20 lines allowing up to 20 patients per page. Some pages might contain more than 20 patients when a nurse has added lines and other cases when the full page has not been used. Do not worry in these cases.
C. Set the sample size n at 30
D. The approximate number of patients covered during the 6 months period is then 20xP=N
E. Calculate the step size k* = [N/n] and Round Down to the nearest integer k
Note:
If P = 30 then one patient record was retrieved per page
If P < 30 then more than one record was retrieved per page
If P > 30 then calculated and followed the step size (some pages might have been skipped)
F. On the first page in the month marking the beginning of the 6 months period (February) a patient from patients 1 to k was randomly selected.
G. k patient records were skipped and the procedure was repeated until the end of the patient records was reached.
H. A record of P, N, n, k, the random start point and other relevant information was kept.
None of the final datasets include all 75 originally targeted facilities; the number of facilities visited per dataset breaks down as follows:
§ The patient exit survey comprises 1,561 interviews at 70 facilities.
§ The staff survey comprises 211 interviews at 66 facilities.
§ The ART cohort administrative data was collected for 1,630 patients at 62 facilities.
§ The outpatient monthly records were collected from 51 facilities.
Three of the 75 health facilities were not included in any of the four datasets; two of these were in Maseru District, which may cause some sampling error. Additionally, the lack of coverage in the outpatient summary records is somewhat worse in Berea and Quthing Districts.
This dataset does not contain weights.
Survey instrument
Exit patient questionnaire: A survey of patients undergoing treatment at health centers in Lesotho. Patients were asked for basic demographic information, details of their travel and expenses to reach the clinic, and their opinion on the facility's amenities and quality of service. The data was collected between August 2010 and January 2011.
Staff questionnaire: A survey of staff at health centers in Lesotho. Respondents were asked for basic demographic information and their rating of their working and living conditions, the quality of the services offered and overall physical state of health facilities. The data was collected between August 2010 and January 2011.
Out-patient Monthly Summary Report: An aggregation of the records from health centers in Lesotho. Data consists solely of the number of adult and child patients of each sex who visited the clinic between February and July of 2010. Data was collected between August 2010 and January 2011.
ART Cohort Study: Data collected from administrative records of patients undergoing anti-retroviral treatments for HIV at sixty-two health centers in Lesotho, consisting largely of patient biodata and monthly treatment types. The data was collected between August 2010 and January 2011.
Data collection
| Start | End | Cycle |
|---|---|---|
| 2010-08-01 | 2011-01-31 | Baseline |
| Name |
|---|
| NORC at the University of Chicago |
The data collection for the exit and staff surveys was conducted by three individuals over a period of six months in 2010. The data collection consisted of four separate parts: two short surveys (a survey of facility staff and an exit interview of patients) and the collection of two types of administrative data (register of patients undergoing anti-retroviral therapy and monthly summary of all outpatients).
For each facility, the collection of all four parts was accomplished either in a single day (concentrating on a single facility), or over two days (at two separate but nearby facilities - one treatment, one control). All data collection was accomplished by a single team of three individuals: two dedicated to the surveys and one in charge of collecting administrative data.
Data processing
The ART register data was lightly edited to resolve formatting inconsistencies: for example, aligning dates to match the same DD.MM.YY format and changing patient sex to either M or F.
The outpatient data included subtotals for adult males, minor males, and minor females, but in some cases, data for adult females was missing. For these, the number of females was calculated by subtracting the sum of the three other categories from the overall total. When this resulted in a negative number, the value was coded as -444. In four facilities, data was missing for one or more months. Village names were also occasionally missing. Finally, there were twelve months in which the four subtotals did not equal the total; in these, the total was changed to reflect the actual sum of the four subtotals.
Access policy
Millennium Challenge Corporation
Millennium Challenge Corporation
https://data.mcc.gov/evaluations/index.php/catalog/163
Cost: None
Data Access
The National Opinion Research Center at the Univeristy of Chicago. 2012. Impact Evaluation Design & Implementation Services - Lesotho Renovation of Health Centers and Outpatient Departments Evaluation.
Metadata production
DDI_LSO_2010_MCC-HCCACI_v01_M
| Name | Role |
|---|---|
| Millenium Challenge Corporation | Metadata Producer |
2016-05-25
Metadata version
Version 2.0 (May 2016). This version adds metadata for the baseline surveys conducted for the evaluation and updates the program logic description.
Version 3 (June 2020). Edited version based on Version 2.0 (DDI-MCC-LSO-HEALTH-NORC-2012-V3) that was done by the Millennium Challenge Corporation.