MNG_2010-2014_MCC-PRPURL_v01_M
Property Rights - Peri-Urban Rangeland Leasing 2010-2014
Independent Impact Evaluation
Name | Country code |
---|---|
Mongolia | MNG |
This metadata file combines three separate evaluations which are part of one entity. They include Land Productivity, Phase I, and Phase II.
Productivity
As part of monitoring and evaluation activities, the Millennium Challenge Corporation (MCC) and Millennium Challenge Account-Mongolia (MCA-M) carried out an impact evaluation that examined how the securing of long-term land use rights and provision of infrastructure and training through PURP affects not just livestock herding efficiency and productivity, but also environmental degradation and rangeland quality in peri-urban areas. In support of the rangeland quality component of the evaluation, MCC signed an agreement with the United States Department of Agriculture (USDA). Under this agreement, USDA provided technical advice and support to MCC regarding the monitoring and evaluation of pastureland and oversaw a local land quality contractor that carried out evaluation activities including the collection of baseline and follow-up data, as well as the training of local technicians and officials in sustainable land quality monitoring.
Methodology: Propensity Score Matching: Randomization
Phase 1
This study is an impact evaluation on the overall effect of the PURP on both project recipients and their neighbors. Key outcomes are household agricultural income, milk yields from cattle, and restriction of herd size to the carrying capacity of the land. A separate but coordinated study will examine the long term effect of the project on land degradation patterns. In Phase 1 areas, eligible applicants to the project went through an extensive screening process, including a requirement to obtain permission from all neighbor households, and all groups that passed through this process were admitted into the project. A comparison group was created through propensity score matching, using households from three populations: 1) applicants who did not pass the full screening process and so were rejected from the project, 2) neighbors of accepted and rejected groups, 3) a random sample of herder households from soums where the project was being implemented.
Methodology: Propensity Score Matching: Difference in Difference
Phase 2
This study is an impact evaluation on the overall effect of the PURP on both project recipients and their neighbors. Key outcomes are household agricultural income, milk yields from cattle, and restriction of herd size to the carrying capacity of the land. A separate but coordinated study will examine the long term effect of the project on land degradation patterns. In Phase 2 areas, eligible applicants to the project were entered into a lottery and half of them were randomly selected to become project beneficiaries. Thus a very clean control group was created to determine the causal impacts of the project.
Methodology: Randomization
Sample survey data [ssd]
Productivity
Individuals, herder groups
Phase 1
Households
Phase 2
Individuals, households
Topic | Vocabulary |
---|---|
Land | MCC Sector |
Productivity
The peri-urban areas included 57 soums and districts located in five aimags and one city. Approximately 387 serviced tracts of pasture land between approximately 500-1500 hectares (grass-fed livestock system) and 100 hectares (intensive system) are being identified for approximately 15 year length project leases.
Peri-Urban Rangeland
Phase 1 of PURP consists of three peri-urban areas, surrounding the three largest cities in Mongolia: Ulaanbaatar, Darkhan, Erdenet.
Phase II of PURP consists of two peri-urban areas. The first is the area surrounding the city of Choibalsan in Dornod aimag, within 45 km of the city. The second is surrounding the cities of Arvaikheer and Kharkhorin in Uvurkhangai and Arkhangai aimags, and consists of parts of 10 soums.
Productivity
Herder groups in intervention areas
Phase 1
Herder households in rural Mongolia, in the areas around Ulaanbaatar, Darkhan, and Erdenet cities.
Phase 2
All households that were members of the 328 herder groups that applied to the PURP in the Phase II areas. And two neighbor households of these groups that lived within 2 km of the lease area boundaries.
Modification for Wave 2:
Sampling of neighbor households was extended to 5 km distance from lease area boundaries, due to low rates of successful neighbor interview in the baseline survey.
Name | Affiliation |
---|---|
Mongolian Society for Rangeland Management (MSRM) | Productivity |
Innovations for Poverty Action | Phase 1 & 2 |
Name |
---|
Millennium Challenge Corporation |
Productivity
The design encompassed a two-stage randomized selection process to determine which Herder Groups will receive the leasing slots that are available for the project. In the first stage of the design, all Herder Groups located in areas deemed fit for the project were allowed to submit applications for the available slots. These applications are cleared as meeting environmental and social assessment criteria and were scored by local selection committees, according to a set of predefined criteria. Those that pass this stage were short-listed for the second stage of selection. In the second stage of selection, the lease slots were randomly assigned to the short-listed candidates. Some candidates were randomly selected to receive a leasing slot (the Treatment group) while other candidates were not (the Control group). Random assignment led to the creation of two virtually identical groups at baseline. The only difference was that the Treatment group was offered the lease and other associated project assistance while the other group (the Control group) was not. As a result, any changes observed between the two groups over time could be attributed to the project.
PHASE I
Baseline
Follow-up
PHASE II
Baseline:
All households that were members of groups that applied to the project were included in the survey (total 923 interviewed). Moreover two neighbor households were intended to be surveyed for each group, randomly chosen from a list provided by a firm that had mapped the households in the areas (total 420 interviewed). For project applicants, power calculations were performed for one key outcome variable, average daily milk yield per milking cow (liters): Baseline value = 2.19, Std Dev. = 1.3, MDE = 0.25
Follow-up (Wave 2):
Sample from baseline was augmented by any PURP treatment and control households that were missed in the baseline survey (total 24).
Additionally, since few groups in the baseline survey had two successful neighbor interviews, neighbors were re-sampled in Wave 2. If a group had one or two neighbors interviewed in the baseline survey, then the same neighbors were to be interviewed in Wave 2. If the group did not have a full set of two neighbors in the baseline, then a list of all neighbor households within ten kilometers of the lease area were given by the herder group leaders. The survey firm then attempted to locate these households. The first two households within 5 kilometers of the lease boundary were to be interviewed. If there were not two households within 5 kilometers, then neighbors up to ten kilometers away could be interviewed. In total 143 new neighbors were added to the sample for Wave 2.
PHASE II
Baseline:
Some groups did not have two neighbors within 2km of their lease area boundary. In these cases sometimes less than two neighbors were interveiwed per group. This was especially the case in the Choibalsan peri-urban area, because households are spread very widely apart.
PHASE I
Baseline: Overall response rate for the Household Survey was 73%
Follow-up: Overall response rate for the Household Survey was 87%
PHASE II
Baseline:
92.6% (Treatment households)
89.1% (Control households)
72.1% (percent of Treatment groups with at least one neighbor interview)
71.7% (percent of Control groups with at least one neighbor interview)
Notes:
Follow-up (Wave 2):
90.3% (Treatment households)
82.4% (Control households)
84.8% (percent of Treatment groups with at least one neighbor interview)
85.2% (percent of Control groups with at least one neighbor interview)
Notes:
Productivity
The PURLS (Peri-Urban Rangeland Leasing Survey) was used to collect data on the behavior and characteristics of Herder Groups in both the Treatment and Control groups. The outcomes of the herder households in the Treatment and Control groups were compared to assess the impacts of the program.
Phase 1
Phase 2
Start | End | Cycle |
---|---|---|
2010-09-03 | 2011-06-14 | Household (Phase 1 - Baseline) |
2011-06-06 | 2011-06-21 | Site selection and collected site characterization data for Phase I |
2011-06-07 | 2011-06-19 | Collected Line-point intercept data for Phase I |
2012-01-30 | 2012-04-21 | Household & Group Leader Baseline (Phase 2 - Wave 1) |
2012-05-03 | 2012-05-21 | Site selection and collected site characterization data for Phase II |
2012-08-14 | 2012-09-09 | Production data for Phases I and II (Fall 2012) |
2012-08-15 | 2012-09-09 | Collected Line-point intercept data for Phase II |
2012-12-10 | 2013-04-26 | Household (Phase 1 - Follow Up) |
2013-04-16 | 2013-05-12 | Production data for Phases I and II (Spring 2013) |
2013-08-17 | 2013-09-13 | Production data for Phases I and II (Fall 2013) |
2014-04-13 | 2014-05-05 | Production data for Phases I and II (Spring 2014) |
2014-05-04 | 2014-08-04 | Household & Group Leader Follow Up (Phase 2 - Wave 2) |
Name | Affiliation |
---|---|
Mongolian Society for Rangeland Management | Productivity |
Joint affiliation of MEC and MCDS LLC | Phase I & Phase II |
PHASE I
Interviewing was conducted by 2 teams of interviewers. Each interviewing team comprised of 1 team leader, 6 interviewers, and 2 drivers. Each team used 2 four wheel drive vehicles to travel from cluster to cluster (and where necessary within cluster).
The role of team leader was to ensure the overall coordination and technical oversight of all the tasks, maintain the communication with IPA and MCA-Mongolia, coordinate with local authorities, and make arrangements for accomodation and travel. Additionally, the team leader was responsible for ensuring the quality of data collection by checking all the paper copy of questionnaires that are filled out by interviewers at the end of everyday.
PHASE II
Interviewing was conducted by 2 teams of interviewers. Each interviewing team comprised of 1 team leader, 6 interviewers, and 2 drivers. Each team used 2 four wheel drive vehicles to travel from cluster to cluster (and where necessary within cluster).
The role of team leader was to ensure the overall coordination and technical oversight of all the tasks, maintain the communication with IPA and MCA-Mongolia, coordinate with local authorities, and make arrangements for accomodation and travel. Additionally, the team leader was responsible for ensuring the quality of data collection by checking all the paper copy of questionnaires that are filled out by interviewers at the end of everyday.
PRODUCTIVITY
Data were collected using the Line-point Intercept method and the Plant Production method. The Plant Production method varied from the standard method in that all plants were clipped at 1cm height from the soil surface, except shrubs and sub- shrubs. Nine plant functional groups were differentiated on each 50cm x 100cm quadrat (subplot). The nine plant functional groups are defined below:
PRODUCTIVITY
Ericha Courtright, the Database for Inventory, Monitoring, and Assessment (DIMA) specialist at the USDA, performed additional data error checking and data cleaning with the help of Justin Van Zee and the MSRM researchers on each set of data after it was collected.
PHASE I
Data editing took place at a number of stages throughout the processing, including:
a) Field check by survey firm team leaders
b) Checking the discrepencies of 1st and 2nd data entry
c) Internal logic check by survey firm
d) Manual data entry check by IPA
e) Logic control check by IPA
f) Data cleaning by IPA
g) Check of outliers by IPA
PHASE II
Data editing took place at a number of stages throughout the processing, including:
a) Field check by survey firm team leaders
b) Checking the discrepencies of 1st and 2nd data entry
c) Internal logic check by survey firm
d) Manual check by IPA
e) Logic control check by IPA
f) Data cleaning by IPA
BASELINE
All surveys were checked for logical consistency by interviewer supervisors in the field, the day they were collected. If a survey contained inconsistent answers, the respondent was re-interviewed to clarify the problematic questions.
FOLLOW-UP
Consistency checking, including skip patterns and restricted value ranges, when applicable, were programmed into the survey software.
Several types of missing values were recorded. These are operationalized as extended missing values in the Stata datasets.
Note: Questions that were accurately skipped (.s) were generally not coded for the baseline survey and were instead left blank. All valid skips were coded in the follow-up survey.
Numeric missing value code is given first (followed by string code in parentheses)
.s (.skip) -- Question was skipped according to a correct skip pattern
.r (.refusal) -- respondent refused to give a valid answer to the question
.d (.don't know) -- respondent did not know the answer to the question
.q (.qre refused) -- respondent refused to give answers to an entire section of the questionnaire
Millennium Challenge Corporation
Millennium Challenge Corporation
https://data.mcc.gov/evaluations/index.php/catalog/211
Cost: None
Is signing of a confidentiality declaration required? |
---|
no |
PRODUCTIVITY
Courtright, E., J. Van Zee, D. James and J. Herrick. 2015. Peri-Urban Rangeland Land Productivity Monitoring and Evaluation Draft. Las Cruces, NM: U.S. Department of Agriculture - Agricultural Research Service Jornada Experimental Range.
PHASE I
Rubenson, Daniel, Erica Field, Leigh Linden, Shing-Yi Wang. 2011. Peri-Urban Rangeland Leasing Survey Phase 1 Baseline (PURLS 1 Baseline). V1.
Rubenson, Daniel, Erica Field, Leigh Linden, Shing-Yi Wang. 2011. Peri-Urban Rangeland Leasing Survey Phase 1 First Follow-up (PURLS 1_W2). V1.
PHASE II
Rubenson, Daniel, Erica Field, Leigh Linden, Shing-Yi Wang. 2012. Peri-Urban Rangeland Leasing Survey Phase 2 Baseline (PURLS 2 Baseline). V1.
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Name | |
---|---|
Monitoring & Evaluation Division of the Millennium Challenge Corporation | impact-eval@mcc.gov |
DDI_MNG_2010-2014_MCC-PRPURL_v01_M
Name | Role |
---|---|
Millennium Challenge Corporation | Metadata Producer |
2015-03-03
Version 1.0 (March 2015)
Version 2.0 (August 2020). Edited version based on Version 1.0 (DDI-MCC-MNG-MULTI-PURP-2018-v1) that was produced by the Millennium Challenge Corporation
2020-08-03