| Table of contents |
List of Acronyms and Abbreviations ....................................................................................................... iii
Executive Summary .................................................................................................................................... v
1. Introduction .......................................................................................................................................... 1
1.1 Country Context ........................................................................................................................... 1
1.2 Report Objective ........................................................................................................................... 8
2. Overview of the Compact and the Interventions Evaluated ............................................................. 9
2.1 Project Logic .............................................................................................................................. 16
2.1.1 Overview of the Project Logic at the Compact Level ................................................... 16
2.1.2 Project Participants ........................................................................................................ 19
2.1.3 Geographic Coverage .................................................................................................... 20
2.1.4 Implementation Summary ............................................................................................. 22
2.2 Cost-Benefit Analysis and Beneficiary Analysis ....................................................................... 26
3. Literature Review ............................................................................................................................... 27
3.1 Summary of the Existing Evidence ............................................................................................ 27
3.2 Evidence Gaps That the Current Evaluation Can Fill ................................................................ 31
4. Evaluation Design ............................................................................................................................... 34
4.1 Overview of Evaluation Design ................................................................................................. 34
4.2 Evaluation Questions .................................................................................................................. 35
4.3 Country-Specific and International Policy Relevance of Evaluation ......................................... 38
4.4 Methodology............................................................................................................................... 38
4.4.1 Quantitative Approach................................................................................................... 38
4.4.2 Qualitative Approach..................................................................................................... 41
4.4.3 Identification Strategy ................................................................................................... 44
4.4.4 Assessment of External Validity Risks ......................................................................... 44
4.5 Sampling Strategy ...................................................................................................................... 44
4.5.1 Power Calculations ........................................................................................................ 44
4.5.2 Construction of the Sample Frame ................................................................................ 45
4.5.3 Sample Selection ........................................................................................................... 45
4.5.4 Balance in Means Tests and Validation of Internal Validity of Evaluation Design ...... 47
4.5.5 Assessment of Internal Validity Risks........................................................................... 47
4.6 Sample Size ................................................................................................................................ 48
4.7 Exposure Period: Includes Exposure Quantity and Duration ..................................................... 49
5. Data Sources and Data Analysis Plan ............................................................................................... 51
5.1 Administrative Data .................................................................................................................... 51
5.1.1 SPSE data Description and data quality issues .............................................................. 51
5.2 Primary Data Collection Protocols ............................................................................................. 54
5.2.1 Quantitative ................................................................................................................... 54
5.2.2 Qualitative ..................................................................................................................... 55
5.3 Data Analysis.............................................................................................................................. 56
5.3.1 Quantitative ................................................................................................................... 56
5.3.2 Qualitative ..................................................................................................................... 57
6. Quantitative and Qualitative Analytic Baseline Findings ............................................................... 58
6.1 Shared Values ............................................................................................................................. 59
6.1.1 Overall Political Economy ............................................................................................ 59
6.1.2 Culture and Shared Values ............................................................................................ 60
6.2 Structure ..................................................................................................................................... 61
6.2.1 Permanency and Independence ..................................................................................... 61
6.2.2 Budget Absorption and Procurement Completion ......................................................... 64
6.3 Systems ....................................................................................................................................... 65
6.3.1 Public Procurement Systems, Policies and Procedures ................................................. 66
6.3.2 Assessment of Procurement Quality and Measures....................................................... 72
6.3.3 Cost Savings Opportunities ........................................................................................... 74
6.3.4 E-catalog and Framework Contracting Awareness and Use ......................................... 75
6.3.5 Awareness of an MIS and Awareness of the New PMIS .............................................. 77
6.3.6 Challenges in the Procurement Process ......................................................................... 78
6.4 Skills ........................................................................................................................................... 78
6.4.1 Required Skills .............................................................................................................. 78
6.4.2 Baseline Knowledge of PSU Staff................................................................................. 79
6.5 Staffing ....................................................................................................................................... 81
6.5.1 Staffing Levels .............................................................................................................. 81
6.5.2 Gender Inclusivity ......................................................................................................... 83
6.5.3 Level of Permanent Staff ............................................................................................... 83
6.5.4 Perception of/Interest in Pursuing a Procurement Career Path ..................................... 84
6.5.5 Staff Retention ............................................................................................................... 85
6.6 Overall Evaluation Questions ..................................................................................................... 86
6.6.1 Perception of Corruption, Bias, Collusion, and Lack of Transparency ......................... 86
6.7 SPSE data Baseline Findings ...................................................................................................... 88
6.7.1 Structure ........................................................................................................................ 89
6.7.2 Systems .......................................................................................................................... 91
6.7.3 Skills .............................................................................................................................. 95
6.7.4 Overall Evaluation Questions ...................................................................................... 101
7. Administrative .................................................................................................................................. 104
7.1 Summary of IRB Requirements and Clearances ...................................................................... 104
7.2 Data Access, Privacy and Documentation Plan ........................................................................ 105
7.3 Dissemination Plan ................................................................................................................... 105
7.4 Evaluation Team Roles and Responsibilities ........................................................................... 105
7.5 Independence Risk Mitigation Approach ................................................................................. 108
7.6 Budget ...................................................................................................................................... 110
8. References ......................................................................................................................................... 113
9. Annexes.............................................................................................................................................. 117
9.1 Research Questions .................................................................................................................. 117
9.2 Power Calculations ................................................................................................................... 120
9.3 Analysis Plan for the Quantitative Data ................................................................................... 122
9.4 Stakeholder Comments and Evaluator Responses .................................................................... 124
9.5 Notes on Finite Population Correction ..................................................................................... 124
9.6 Quantitative Analytic Results Disaggregated by Gender ......................................................... 125
9.7 Procurement Skills Quiz Survey Module ................................................................................. 128 |
