Threshold 2014

The mixed methods evaluation was carried out by a four-member team from Social Impact, Inc. during field visits in early 2014. In February 2014, while the FOTI project was still active, the Team Leader completed preliminary data collection during an evaluation design trip. In July 2014, the other three team members completed data collection during a two week field visit to Timor-Leste. The evaluation utilized ongoing data analysis, an iterative methodology, and careful triangulation of data sources throughout the research process. The evaluation team utilized the following methodologies: document review; technical evaluation of "deliverables"; pre- and post-test data; key informant interviews; and a field visit to Baucau.

Although the evaluation team attempted to complete a rigorous examination of the approved research questions, it proved impossible to utilize all of the proposed research methodologies due to logistical and bureaucratic realities in Timor-Leste. For example, although the team initially planned to conduct mini-surveys and focus group discussions, the team did not gain access to enough program beneficiaries to make data obtained from such informants meaningful. Other limitations include: challenge in measuring qualitative outcomes; logistical and timing challenges; recall and attribution bias; selection bias; and use of interpreter. For further details, please see Annex II. While these factors limit the quantity and quality of the data collected, the team made every effort to fill gaps in the data, particularly by examining all available program documentation. Through this triangulation of sources, the team feels that the information presented in this report is an accurate depiction of the FOTI program.