World Bank Group Impact Evaluations: Relevance and Effectiveness

by World Bank

Overview

Impact evaluation has grown more popular as a method for identifying the causal links between interventions and outcomes. These kind of evaluations assess changes that can be attributed to a particular intervention. Both innovations in statistical methods and the demand for evaluations that can measure such development results are increasing. The World Bank Group is the largest producer of impact evaluations among all development institutions. Thus, IEG has evaluated the relevance, quality, and influence of World Bank and IFC impact evaluations. IEG finds that the World Bank Group portfolio of impact evaluations is largely aligned with sector strategies and project objectives. Selection and coordination of impact evaluations has been improving. Most World Bank impact evaluations meet either medium or high quality standards, and about half of IFC impact evaluations did. Issues related to funding, staff capacity, and incentives, however, constrain the scope and coverage of impact evaluations in the Bank Group. IEG makes five recommendations to strengthen the Bank Group’s impact evaluation efforts, revolving around consistency, coordination, quality standards, and ensuring operational relevance. Both development and evaluation professionals will find valuable lessons in this evaluation. There are real benefits from impact evaluations, including their influence on development practices through contributions to project assessment and design of future projects. Thus, development practitioners engaged in designing projects, evaluators interestedin using similar methodology, and the general evaluation community will be able to use the lessons IEG sets out in this report.

Product Details

ISBN-13:	9780821397176
Publisher:	World Bank Publications
Publication date:	03/12/2013
Series:	Independent Evaluation Group Studies
Pages:	200
Product dimensions:	6.80(w) x 10.10(h) x 0.60(d)

Table of Contents

Acknowledgments ix

Overview xi

Management Response xxvii

Chairperson's Summary: Committee on Development Effectiveness xxxix

Abbreviations xl

1 Introduction 1

What Is Impact Evaluation? 3

Evaluation Objective, Scope, and Framework 5

Building Blocks of the Evaluation 9

Notes 11

References 13

2 Impact Evaluations at the World Bank Group 15

Institutional Efforts to Facilitate Production of Impact Evaluations 17

Production of World Bank Group Impact Evaluations 23

Thematic and Regional Distribution of World Bank Group Impact Evaluations 26

World Bank Group Impact Evaluation Methods 30

Stakeholder Involvement in World Bank Group Impact Evaluations 31

Costs and Funding Mechanisms for World Bank Group Impact Evaluations 33

Notes 36

References 40

3 Relevance of World Bank Group Impact Evaluations 43

Relevance of Impact Evaluations to Operational Needs 46

Relevance of Impact Evaluations to Institutional Strategies 49

Relevance of Impact Evaluations for Knowledge Generation 53

Factors Affecting Scope and Relevance of World Bank Group Impact Evaluations 55

Notes 64

References 72

4 Quality of World Bank Group Impact Evaluations 75

Quality Review Methodology 78

Data Quality and Outcome Indicators 79

Ability to Infer Causal Effects 82

Factors Associated with the Quality of World Bank Group Impact Evaluations 85

Notes 88

References 89

5 Use and Influence of World Bank Group Impact Evaluations 91

Influence of Impact Evaluations on Project Operations 93

Influence of Impact Evaluations on Policy Dialogue 98

Influence of Impact Evaluations on World Bank Group Strategies 100

Influence of Impact Evaluations on Evaluation Capacity and Culture 102

Factors Affecting Use and Influence of Impact Evaluations 106

Notes 112

References 115

6 Conclusions and Recommendations 117

Conclusions 118

Recommendations 121

Appendixes

Appendix A Glossary of Terms 125

Appendix B Methodology 133

Appendix C Assessing the Potential Coverage of Impact Evaluations in Education 143

Appendix D Analysis of Impact Evaluation and Intervention Costs 149

Appendix E Criteria for Scoring Spanish Trust Fund for Impact Evaluation Cluster Proposals 153

Appendix F Quality Review Framework 155

Appendix G Conceptualizing Use and Influence of Impact Evaluation 161