Green Prosperity: Participatory Land Use Planning 2016
Indonesia, 2016
Get MicrodataIdentification
IDN_2016_MCC-GPPLUP_v01_M
Green Prosperity: Participatory Land Use Planning 2016
| Name | Country code |
|---|---|
| Indonesia | IDN |
Independent Performance Evaluation
This evaluation is composed of two components, a pre-post performance evalaution (PE) of PLUP pilot ditricts implmented in two stages and a PE of PLUP expansion districts implemented in Stage 2.
Stage 1 of data collection took place during latter half of implementation of PMAP 1 implementation in the four pilot districts, in August - September 2016 (and was informed by a scoping trip in January 2015). It forms an early results analysis for the pilot districts and comprises a full assessment of the original evaluation questions.
Stage 2 of data collection is informed by the results from the Stage 1 of data collection and a scoping trip conducted in July 2019. Data collection will take place approximately two years after the closure of the Compact (2020) in both pilot districts and the expansion districts, of which there are 40 in total. In pilot districts, following the first stage of data collection, the evaluation team will focus on the full range of evaluation questions. In expansion districts, the evaluation will focus on a meso-level assessment of some of the bigger questions about the continuity of PLUP outcomes after the completion of the project, with less emphasis on outcomes for households or individual community members and more on the efficacy and sustainability of the systems that PLUP implemented. This means that expansion sites will not conduct in depth inquires at the village level to understand perceptions and experiences of community members but focus more on the influence that the land management systems imparted by PLUP have affected decision making, land use allocation, and larger-scale results.
Similar to Stage 1, Stage 2 of the evaluation will use a mixed-method approach, with primary qualitative data collection and analysis of available quantitative secondary data. Qualitative data collection will include a thorough review of selected PMAP narrative report monitoring data, as well as key informant interviews and focus group discussions (primary qualitative data collection) with project stakeholders across multiple levels of selected PMAPs. Different than Stage 1, Stage 2 will have a more robust quantitative data analysis component, which will include geospatial data secondary data collected by government actors, and key datasets (discussed in detail below) that could contribute to an analysis of PLUP outcomes. Findings will be triangulated against data accessed through public record and in consultation with provincial and district government offices.
Evaluation questions at Stage 2 have been updated from the Stage 1 original questions as follows:
EQ1. Original: How has PLUP progressed in the short-term outcomes, and how is it likely to progress in the achievement of long-term outcomes?
Revised (Expansion and Pilot sites): To what extent have a) PLUP outputs been sustained and b) short- and medium-term outcomes been realized?
EQ2. Original (retained for Pilot sites): To what extent did the sustainability of outputs and achievement of outcomes vary by geography or sex and vulnerable/ marginalized groups?
Revised (added for Expansion sites): To what extent did the sustainability of outputs and achievement of outcomes vary by implementer, contract type (with or without Task 1), geography/land use, or sex and vulnerable/marginalized groups?
EQ3. Original: What were the main challenges in managing PLUP?
Revised (Expansion sites): To what extent did PLUP lessons learned and changing environmental factors impact implementation of expansion site PMAP contracts? What were the main challenges in managing and implementing PLUP?
Revised (Pilot sites): Not applicable.
EQ4. Original (Retained for both Pilot and Expansion sites): What were unintended results (positive or negative) achieved by PLUP?
EQ5. Original (REMOVED): Through what pathways, if any, is increased spatial certainty likely to increase household incomes? What evidence does the evaluation find of this?
The Evaluation Summary will be updated with the final evaluation results at the completion of Stage 2.
Other
For pilot districts, Task 1 VBS/RM related outcomes, the unit of analysis is citizens or villagers.
For other outcomes in Tasks 2-4, the unit of analysis is institutional stakeholders (primarily at the district level, but also at the provincial and national levels, to a lesser extent).
For expansion districts, the evaluation will focus on a meso-level assessment of some of the bigger questions about the continuity of PLUP outcome and the unit of analysis is institutional stakeholders and investors at the district, provincial and national levels.
Scope
| Topic |
|---|
| Participatory Mapping |
| Participatory Land Use Planning |
| Village Boundary Setting |
Coverage
For pilot districts, data were sampled purposively, and thus cannot be considered representative at any level. The results will not be generalizable to the PMAP 2 - 8 districts (expansion districts). This is due to the sampling of districts for this evaluation, and to the fact that most of the PMAP contracts do not implement Task 1 - 4 as PMAP 1 does. However, data were collected from the four districts covered under PMAP 1 (from six of the eight sub-districts) as well as from provincial and national stakeholders. These four districts are Merangin and Muaro Jambi in Jambi province and Mamasa and Mamuju in West Sulawesi Province.
For the 40 expansion districts, 100 percent coverage of the districts is not feasible within the budget for this evaluation and therefore sampling is required at least for qualitative data collection. Sampling for quantitative datasets may also mirror qualitative sample districts if the datasets are not available at the national level and have to be collected at district levels. The BAPPENAS MCA-I secretariat suggested that the evaluation cover each island region, while also considering dominant land use types (e.g. plantations, logging, mining, tourism). The sampling approach for expansion districts followed this guidance, while also seeking to cover all PMAP contractors and work within the allotted budget. Districts which will be visted for qualitative data collection include Kolaka and East Kolaka in Southeast Sulawesi, Rokan Hulu and Pelalawan in Sumatra and Ende and Sikka in East Nusa Tenggara.
The study population includes all participants and implementers of PLUP.
Producers and sponsors
| Name |
|---|
| Social Impact |
| Name |
|---|
| Millennium Challenge Corporation |
Sampling
The approach to sampling frame development for each respondent type (KIIs, FGDs) is consistent in both pilot and expansion sites at Stage 1 and 2. Government stakeholders are purposefully selected based on responsibilities relevant to PLUP. In expansion districts, village stakeholders will be restricted to leaders for expansion districts while being broader in pilot sites. Leaders may broadly be defined as the village leader and customary leaders but could also include leaders of a village women's or youth group, or committee that oversees planning. Investors will be selected by snowball sampling from relevant government agencies and/or business associations, or from a list of businesses that have obtained permits since the end of PLUP implementation if made available through district government agencies.
This sample is not meant to be representative, and no power calculations were conducted since the data are mostly qualitative.
During Stage 1 the original plan was to conduct 110 KIIs and 8 FGDs. When the team was in the districts, however, there was excitement and high levels of engagement from related government offices. This resulted in more FGDs and fewer KIIs.
Not applicable.
Not applicable.
Survey instrument
All questionnaires were written in English and then translated into Bahasa Indonesia. The 12 utilized questionnaires for Stage 1 are detailed briefly below:
- KII-Village Member: A VPT member or Dispute Forum member (if not village officials), or another community leader/figure. This guide may be used for individual interviews or group interviews (with two to three individuals that have similar roles in the village such as small scale farmers or members of women’s groups or PKK).
- KII-Village Official: A village official/leader (who can also be a VPT or dispute forum member).
- KII-District Level: Separate guides used for: (i) BAPPEDA staff members identified by PMAP 1, and when possible staff members that directly participated in PMAP 1 activities for Tasks 2 – 4; (ii) OSS staff members identified by PMAP 1, and when possible staff members that directly participated in PMAP 1 activities for Tasks 2 – 4; or (iii) The Camat (Sub-district head).
- KII-Provincial Officials: Officials from provincial BAPPEDA, BPN, etc.
- KII-National Officials: Bappenas, BIG, BRG, and MOHA
- KII-MCC and MCA-I Staff: Respondents to include GP Management (Jakarta), PLUP MCA-I Management Team (Jakarta), MCAI M&E Team (Jakarta), MCC PLUP Advisors (Jakarta and USA), MCAI Window Program Managers, MCAI Communications Team, MCAI Economics team, and PLUP-related consultants (when available, Jakarta).
- KII-PMAP 1 Implementers: Respondents to include PMAP 1 COP, PMAP 1 Jakarta staff (including Task Coordinators 1 – 4), District Team Leaders, CLCS, and SCF (when and where available). Respondents will also include Warsi and Puter staff where they fill the positions mentioned previously. Respondents will also include Trimble staff that are visiting Indonesia during the time of the evaluation.
- KII-NGO/CSO/Research Institutions
- KII-Concessionaire/Land Claimant: Concessionaires/Land Claimants (at the national, provincial and district level, as available).
- FGD-Village Level Community Members: A VPT or Dispute Forum of five to six individuals, or a group of community members if the VPT or Dispute Forum does not exist or is not available.
- FGD-Land Claimants/Concessionaires/Businesses: Concessionaires/Land Claimants (at the national, provincial and district level in groups of five or six (as available).
- FGD-Training Beneficiaries: Participants of either the IMS or GIS training conducted by PMAP 1. Trainees are typically from BAPPEDA and the OSS in the districts.
Stage 2 questionnaires, particularly for expansion districts, are under development.
Data collection
| Start | End |
|---|---|
| 2016-08-29 | 2016-09-30 |
At Stage 1 the data collection team included four team members in Jambi, with a fifth team member joining in West Sulawesi who was fluent in the local language. The team travelled and conducted data collection together, switching responsibilities between asking questions and taking notes. All team members shared responsibility for transcribing and note taking post-interview. The evaluation specialist was responsible for checking accuracy of information and applying the coding scheme.
Stage 2 team composition will be updated upon the evaluation's completion.
At Stage 1 each interview team consisted of two or more interviewers, with a female interviewer present whenever a female respondent was present. The team was supported by local drivers and local implementing staff in each district. A total of five interviewers were used for data collection. Data collection took place from August 29 - September 30, 2016. Two weeks were spent in Jakarta and the remaining weeks were spent across two provinces (two districts and six sub-districts). Interviewing took place every day of the week. Interviews averaged 1 hour for all questionnaires - most all respondents were willing and available to talk for over an hour. Interviews were all conducted in Bahasa Indonesia except with select individuals at the national level.
Stage 2 data collection is expected to largely mirror Stage 1 format.
Data processing
For Stage 1 interview notes were cleaned at the end of each day of data collection, and aggregated at the end of each week in the evaluation team's data management system. In the event that the quantity of notes to clean compromised the team's ability to do so meticulously, the evaluation team hired a transcriber to keep up and ensure high quality data. All data editing was conducted manually based on end of day discussions between team members to clarify inconsistencies between notes. The team conducted team analysis sessions two-three times per week to help identify emerging themes, trends, and/or findings. After the team completed data collection, cleaned interview notes were printed for team analysis. In addition, the evaluation specialist applied a total of 28 codes to the interviews, FGDs, and observations to better organize and document themes related to evaluation questions.
Stage 2 is expected to largely mirror Stage 1.
Data appraisal
Not applicable.
Access policy
Millennium Challenge Corporation
https://data.mcc.gov/evaluations/index.php/catalog/180
Cost: None
Data Access
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
EDR 1: Fisher, M., Millette, C., & Stek, A. Participatory Land Use Planning Performance Evaluation Design Report. Social Impact Inc. 2017.
Stage 1 Evaluation Report: Fisher, M., Millette, C., & Stek, A. Participatory Land Use Planning (PLUP) Evaluation: Round 1 Report. Social Impact Inc. 2017.
EDR 2: Adam, U., Fisher, M., Myers, R., & Stek, A. Participatory Land Use Planning Performance Evaluation Design Report (Updated 2019). Social Impact Inc. 2019.
Contacts
| Name | |
|---|---|
| Monitoring & Evaluation Division of the Millennium Challenge Corporation Monitoring & Evaluation Division of the Millennium Challenge Corporation | impact-eval@mcc.gov |
Metadata production
DDI_IDN_2016_MCC-GPPLUP_v01_M
| Name | Role |
|---|---|
| Millennium Challenge Corporation | Review of Metadata |
2016-10-06
Metadata version
Version 1 (2016-10-6): This is the first metadata entry for the PLUP evaluation. It has been developed on the basis of the Evaluation Design Report.
Version 2 (2020-01-14): This is the second metadata entry for the PUP evaluation. It has been developed on the basis of revisions to the Evaluation Design Report in December 2019.
Version 3 (June 2020) edited version based on version 2 (DDI-MCC-IDN-PLUP-2020-v01) that was produced by the Millenium Challenge Corporation.
Given the nature of the project to provide strengthening in technical information and practices in spatial planning and governance capacity strengthening, the stakeholders range across national, provincial, district, sub-district and village levels. Participants differ in their proximity and relationship to the project. These details are included below, by level.
National: National level stakeholders are largely partners, including BAPPENAS and the Ministry of Home Affairs. They did not receive direct training from the project, though representatives from Badan Informasi Geospatial (BIG) did receive project outputs (related to Task 4).
Provincial: Provincial level government offices were not targeted by the program, though they are indirectly related to the work of the project. In particular, these stakeholders include BAPPEDA and the Regional Investment and Coordinating Board (BKMPD/A).
District: At this level, the project did have direct program participants. Depending on the district, members of district government offices were invited to participate in IMS and GIS trainings. Participants were selected by the government offices themselves after training invitations were sent by the project, including from BAPPEDA, Sekretariat Daerah including Asisten Daerah, Forestry Office, Cash-Crops Office, One-Stop Permitting Office (Perijinan Terpadu Satu Pintu), Land Office (Kantor Pertanahan), and Mines and Energy Office (among others). Other district level participants in the project included TPPBD members, selected/appointed by the initial steps of the VBS/RM process (Task 1). These government staff were from the office of Assisten 1 or other agencies.
Sub-District: The sub-district head (Camat) was a participant in the settling of village boundary disputes and in the VBS/RM process.
Village: Village participants were selected through the VPT selection process, as stipulated by MCAI. There were five members of each VPT in 114 villages across the project. In all but two cases, the VPT had one woman member. These villagers were appointed to the VPT role. The village head (Kepala Desa) also participated in the VBS/RM process through dispute resolution and review of outputs.