Systematic Regularization of Land in Urban and Peri-Urban Areas 2013-2019, Independent Impact and Performance Evaluation
Independent Impact and Performance Evaluation
The Millennium Challenge Corporation (MCC) established a partnership with Michigan State University (MSU) to design and conduct the evaluation of the Land Administration Reform Project (LARP). A matched comparison group difference-in-differences evaluation strategy was designed and baseline data were collected in March-June 2013 (https://data.mcc.gov/evaluations/index.php/catalog/85), aimed to test whether the following expected outcomes were realized and attributable to LARP:
(1) Reduction in the financial and time burden of conducting land transaction with LAA and increased efficiency in rendering land administration services to the public by LAA;
(2) Reduction in time for land conflict resolution and reduction in land related conflicts within the areas where there has been intervention amongst the 55,000 lease holders;
(3) Increased number of land parcels used as collateral for mortgage, and increased property investment, subleasing, rentals and other economic activities;
(4) Increased frequency of formal land transaction, increased land values, and increased base case mortgage lending volume;
(5) Increased household income of primary and secondary beneficiaries;
(6) Increased understanding by Basotho of their rights and knowledge about services rendered by the LAA; and
(7) Increased willingness of other land owners outside the regularization impact areas to request formal land title.
After reviewing the initial evaluation design and baseline data, complementary approaches are proposed for impact and performance evaluation of LTRP activities. Besides conducting follow up survey of households interviewed at baseline, geographic discontinuity design (households sampled on both sides the border of treatment wards) and use of time-series high resolution aerial photography are proposed to cost effectively complement the original methodology and assess the medium-term effect of the different components of LARP. The follow up survey is conducted by the Lesotho Bureau of Statistics and extraction of vector data from 2009 and 2016 high resolution data is done by the World Bank research team. Anonymized clean data will be publicly available once data collection and compilation are completed.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Households, properties/parcels, individuals
- v01: Edited, anonymous dataset for public distribution.
The survey covered the following village/sub-villages in MMC1, MMC2, MMC3 and MMC27 in Maseru City
1904 households in 40 clusters at the baseline in Maseru
Producers and sponsors
Millennium Challenge Corporation
The first evaluation design used a randomized control trial (RCT) methodology with a two-stage cluster sampling design to select the sampled households' details of which can be found in the baseline report. This impact evaluation strategy designed at baseline had several problems that needed to be addressed for any results derived from an end-line survey to be credible. The main ones are: (i) non-comparability of treatment and control areas; (ii) implementation of similar interventions in the control areas; and (iii) little consideration of information spillovers.
Suggested approaches to address these challenges include:
I) Propensity score matching (PSM): Baseline data shows significant differences in observed characteristics between treatment and control groups. These differences threaten the internal validity of the evaluation. Use of propensity score matching (PSM) design will thus help attenuate selection bias by increasing the balance between treatment and control groups. Conducting the PSM after the baseline helps reevaluate the statistical power of the sample and guide the data collection for the endline survey.
After dropping households with lease certificates and those with missing data, only 1,418 households (931 from treatment and 487 from control) with valid observations remain for the estimation of the PSM model.
The matching exercise is done at the household level, and covariates at the parcel and individual level are aggregated at the household level. The 5-nearest neighbors matching model was estimated using the STATA program psmatch2. Many observable characteristics at individual, household and parcel level (50 covariates in total) can help capture observed differences between treatment and control groups. Overall, there are 108 off-support treated households and 35 unmatched control households with the 5-nearest neighbor matching criteria. The total matched sample size is thus 1,275 households (823 treatment and 452 control households).
The parameters consist of: (i) 40 clusters (28 treatment and 12 controls); (ii) matched sample size of 1,275 households; and (iii) 31.875 average number of households per cluster, 122.4 individuals with valid employment questions per cluster and 38.825 parcels of land per cluster.
II) Geographic discontinuity design: The discontinuity design heavily relies on the assumption that households located near (on both sides of) the road dividing treatment and control area were similar in observed and unobserved characteristics before the start of the land regularization program. Visual assessment of the 2012 aerial photo shows continuity in the density and type of structure of buildings in most parts across both sides of the road. This observation makes geographic discontinuity design with alternative bandwidth (i.e., comparing all units within a fixed distance from the road), as shown in figure 4 with examples of 500-meter and 1000-meter bands, a viable option to supplement the non-randomized design.
Administrative data show that there are close to 1200 parcels that were under LARP in the treated 500m band from the road. Assuming continuity in settlement in the corresponding part of the control area points towards a rate of contamination of 10% (for sporadic processes) and 30% (for the 2016 layout plan and surveying exercise by Maseru City Council). For a total sample size of 1702 or 2000, the minimum detectable effects, on average, will be 40% or 50% less than that required for the matching design. This suggests that a complete census of households within 500-meter band of the road dividing treatment and control area can provide a sample size large enough to detect meaningful impacts of the program.
III) Analysis of time series aerial photos: High resolution aerial photography of our area of interest are available for two years covering pre-program (2012) and post-program (2016) period. This approach will allow identification of land use changes with a focus on changes in built-up area and possibly building characteristics (e.g. better roofs or pavement of walkways and yards) in treatment and control area. Examples of the type changes (based on 2012 and 2016 photos) that can be easily detected using aerial photography include: (i) changes in roof types (color or shape); (ii) expansion of built-up area; (iii) construction of new houses (either after demolishing existing structure or conversion of vacant plots); and (iv) completing unfinished structures.
Like the geographic discontinuity design, different bandwidths can be constructed around the main road to increase pre-program comparability between the treatment and control area. Besides, it will allow us to identify and characterize locations in which property development have been taking place during the post-intervention period. Given data will be generated for pre- and post-program periods for the treatment and control area, a difference-in-differences (DID) specification will be used to assess the impact of the program particularly on housing-related investment.
IV) Performance evaluation using administrative data: In addition to the expected impacts at the household level, the LARP legal, regulatory and institutional reforms would help facilitate overall land governance in Lesotho by enhancing the registration of land and mortgage transactions, property development investments, land utilization and urban development, land dispute resolution and gender equality. To conduct such type of performance evaluation, administrative and qualitative data are compiled from several sources. The main ones are: (i) spatial and textual land administration data (including mortgage registration) from the Lesotho Land Administration Authority (LAA) and (ii) data on incidence of building permits from Maseru City Council. Analysis of the administrative data will help answer the following main questions: (i) gender equality in terms of access to residential land and rights before and after the program as well as systematic versus sporadic registration; (ii) duration to registered land and mortgage transactions before and after the establishment of LAA; (iii) incidence of land-related disputes over time since the establishment of LAA; and (iv) issuance of building permits before and after the establishment of LAA.
Dates of Data Collection
Data Collection Notes
I) Primary Data Collection
Instruments: The endline survey will revisit households that had been included in the 2013 baseline survey. The survey instruments (both from household head and women respondents) for the endline survey will be the same as the baseline survey with slight modification to cover changes in the dynamics of household composition and landownership as well as participation in the land tenure regularization program.
A short version of the endline survey instruments with a focus on housing and land-related investments will be administered for the geographic discontinuity design.
II) Secondary Data
The research team has access to high resolution aerial photos taken in 2002 and 2016 covering the entire Maseru city. The images will be processed to extract information, at least, at a parcel level on: (i) changes built up area, (ii) changes in roof types, (iii) changes in investments such as fences and improvements to home access, and (iii) access to infrastructure.
All primary data are electronically collected using Survey Solutions deployed on a highly secure World Bank Cloud with geographic questions capable of capturing parcel boundaries using high resolution imagery as a basemap.
Complementing the baseline questionnaire, the endline questionnaire consists of over 20 sections with modules on:
· Identifying household members at baseline/new members
· Household characteristics (demographic information by each member of the HH)
· Employment and sources of any other cash transfers
· Identification and list of all the parcels
· Information on Parcel Acquisition, Documents, Land Value
· Land conflicts
· Rights to the land and perceptions of the risk
· Parcels rented out, rented in
· Characteristics of parcels
· Investments on land
· Perceptions about Lease, renting land, the land law, women's rights and LAA
· Ownership of Assets
· Credit in the last 12 months
· Woman module
Millennium Challenge Corporation
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download