Systematic Regularization of Land in Urban and Peri-Urban Areas 2013-2019, Independent Impact and Performance Evaluation

The first evaluation design used a randomized control trial (RCT) methodology with a two-stage cluster sampling design to select the sampled households' details of which can be found in the baseline report. This impact evaluation strategy designed at baseline had several problems that needed to be addressed for any results derived from an end-line survey to be credible. The main ones are: (i) non-comparability of treatment and control areas; (ii) implementation of similar interventions in the control areas; and (iii) little consideration of information spillovers.

Suggested approaches to address these challenges include:

I) Propensity score matching (PSM): Baseline data shows significant differences in observed characteristics between treatment and control groups. These differences threaten the internal validity of the evaluation. Use of propensity score matching (PSM) design will thus help attenuate selection bias by increasing the balance between treatment and control groups. Conducting the PSM after the baseline helps reevaluate the statistical power of the sample and guide the data collection for the endline survey.
After dropping households with lease certificates and those with missing data, only 1,418 households (931 from treatment and 487 from control) with valid observations remain for the estimation of the PSM model.
The matching exercise is done at the household level, and covariates at the parcel and individual level are aggregated at the household level. The 5-nearest neighbors matching model was estimated using the STATA program psmatch2. Many observable characteristics at individual, household and parcel level (50 covariates in total) can help capture observed differences between treatment and control groups. Overall, there are 108 off-support treated households and 35 unmatched control households with the 5-nearest neighbor matching criteria. The total matched sample size is thus 1,275 households (823 treatment and 452 control households).
The parameters consist of: (i) 40 clusters (28 treatment and 12 controls); (ii) matched sample size of 1,275 households; and (iii) 31.875 average number of households per cluster, 122.4 individuals with valid employment questions per cluster and 38.825 parcels of land per cluster.

II) Geographic discontinuity design: The discontinuity design heavily relies on the assumption that households located near (on both sides of) the road dividing treatment and control area were similar in observed and unobserved characteristics before the start of the land regularization program. Visual assessment of the 2012 aerial photo shows continuity in the density and type of structure of buildings in most parts across both sides of the road. This observation makes geographic discontinuity design with alternative bandwidth (i.e., comparing all units within a fixed distance from the road), as shown in figure 4 with examples of 500-meter and 1000-meter bands, a viable option to supplement the non-randomized design.
Administrative data show that there are close to 1200 parcels that were under LARP in the treated 500m band from the road. Assuming continuity in settlement in the corresponding part of the control area points towards a rate of contamination of 10% (for sporadic processes) and 30% (for the 2016 layout plan and surveying exercise by Maseru City Council). For a total sample size of 1702 or 2000, the minimum detectable effects, on average, will be 40% or 50% less than that required for the matching design. This suggests that a complete census of households within 500-meter band of the road dividing treatment and control area can provide a sample size large enough to detect meaningful impacts of the program.

III) Analysis of time series aerial photos: High resolution aerial photography of our area of interest are available for two years covering pre-program (2012) and post-program (2016) period. This approach will allow identification of land use changes with a focus on changes in built-up area and possibly building characteristics (e.g. better roofs or pavement of walkways and yards) in treatment and control area. Examples of the type changes (based on 2012 and 2016 photos) that can be easily detected using aerial photography include: (i) changes in roof types (color or shape); (ii) expansion of built-up area; (iii) construction of new houses (either after demolishing existing structure or conversion of vacant plots); and (iv) completing unfinished structures.
Like the geographic discontinuity design, different bandwidths can be constructed around the main road to increase pre-program comparability between the treatment and control area. Besides, it will allow us to identify and characterize locations in which property development have been taking place during the post-intervention period. Given data will be generated for pre- and post-program periods for the treatment and control area, a difference-in-differences (DID) specification will be used to assess the impact of the program particularly on housing-related investment.

IV) Performance evaluation using administrative data: In addition to the expected impacts at the household level, the LARP legal, regulatory and institutional reforms would help facilitate overall land governance in Lesotho by enhancing the registration of land and mortgage transactions, property development investments, land utilization and urban development, land dispute resolution and gender equality. To conduct such type of performance evaluation, administrative and qualitative data are compiled from several sources. The main ones are: (i) spatial and textual land administration data (including mortgage registration) from the Lesotho Land Administration Authority (LAA) and (ii) data on incidence of building permits from Maseru City Council. Analysis of the administrative data will help answer the following main questions: (i) gender equality in terms of access to residential land and rights before and after the program as well as systematic versus sporadic registration; (ii) duration to registered land and mortgage transactions before and after the establishment of LAA; (iii) incidence of land-related disputes over time since the establishment of LAA; and (iv) issuance of building permits before and after the establishment of LAA.