This book addresses the challenges of conducting program evaluations in real-world contexts where evaluators and the agencies face budget and time constraints and where critical data is missing. The book is organized around a seven-step model developed by the authors, which has been tested and refined in workshops. Vignettes and case studies—representing evaluations from a variety of geographic regions and sectors—demonstrate adaptive possibilities for small projects with budgets of a few thousand dollars to large-scale, long-term evaluations. The text incorporates quantitative, qualitative, and mixed-method designs and this Second Edition reflects important developments in the field over the last five years.
“This book represents a significant achievement. The authors have succeeded in creating a book that can be used in a wide variety of locations and by a large community of evaluation practitioners.”
“This book is exceptional and unique in the way that it combines foundational knowledge from social sciences with theory and methods that are specific to evaluation.”
“The book represents a very good and timely contribution worth having on an evaluator’s shelf, especially if you work in the international development arena.”
Mara L. Schoeny
“Best text I’ve found for introducing evaluation design AND critical thinking, especially as it relates to current and generally accepted practice, as well as the crucial grounding in real world concerns.”
Michael Bamberger has almost 40 years of experience in development evaluation, including a decade working with nongovernmental organizations in Latin America, almost 25 years working on evaluation with the World Bank in most of the social and economic sectors and in most regions of the world, and 10 years as an independent evaluation consultant, including programs with 10 United Nations agencies and multilateral and bilateral development agencies. He has published three books and several monographs and handbooks on development evaluation, as well as numerous articles in professional journals. He has been active for 20 years with the American Evaluation Association, serving on the Board and as Chair of the International Committee. He has served on the Editorial Advisory Board of New Directions for Evaluation, the Journal of Development Effectiveness, the Journal of Mixed Methods Research, and the American Journal of Evaluation and is a regular reviewer for several professional evaluation journals. He has taught program evaluation in more than 30 countries in Africa, Latin America, Asia, and the Middle East and, since 2002, has been on the Faculty of the International Program for Development Evaluation Training (IPDET) in Ottawa, Ontario, Canada; since 2001, has also lectured at the Foundation for Advanced Studies on International Development (FASID) in Tokyo.
Jim Rugh has had 41 years of professional involvement in rural community development in Africa, Asia, and Appalachia. He has specialized in evaluation for 25 years—the past 10 years as head of Design, Monitoring and Evaluation for CARE International, a large nongovernmental organization (NGO). His particular skills include promoting strategies for enhanced capacity for evaluation throughout this worldwide organization. He is a recognized leader in evaluation among colleagues in the international NGO community, including Inter Action. He has been an active member of the American Evaluation Association since 1986, currently serving on the Nominations and Election Committee. He was a founding member of the Atlanta-area Evaluation Association. He has experience in promoting community development and evaluating and facilitating self-evaluation by participants in such programs. He has provided training for and/or evaluated many different international NGOs. He brings a perspective of the “big picture,” including familiarity with a wide variety of community groups and assistance agencies in many countries, plus an eye to detail and a respect for inclusiveness and the participatory process.
Linda Mabry is a faculty member at Washington State University specializing in program evaluation, student assessment, and research and evaluation methodology. She currently serves as president of the Oregon Program Evaluation Network and on the editorial board for Studies in Educational Evaluation. She has served in a variety of leadership positions for the American Evaluation Association, including the Board of Directors, chair of the Task Force on Educational Accountability, and chair of the Theories of Evaluation topical interest group. She has also served n the Board of Trustees for the National Center for the Improvement of Educational Assessments and on the Performance Assessment Review Board of New York. She has conducted evaluations for the U.S. Department of Education, National Science Foundation, National Endowment for the Arts, the Jacob Javits Foundation, Hewlett-Packard Corporation, Ameritech Corporation, ATT-Comcast Corporation, the New York City Fund for Public Education, the Chicago Arts Partnerships in Education, the Chicago Teachers Academy of Mathematics and Science, and a variety of university, state, and school agencies. She has published in a number of scholarly journals and written several books, including Evaluation and the Postmodern Dilemma (1997) and Portfolios Plus: A Critical Guide to Performance Assessment (1999).
Part 1. The Real World Evaluation Approach
Chapter 1. Overview: Real World Evaluation and the Contexts in Which It Is Used
Chapter 2. [Step 1] Scoping the Evaluation: First clarify the purpose
Chapter 3. [Step 2] Budget Constraints: Not enough money
Chapter 4. [Step 3] Time Constraints: Addressing scheduling and other time constraints
Chapter 5. [Step 4] Data Constraints: Critical information is missing or difficult to collect
Chapter 6. [Step 5] Addressing Political Influences: Reconciling different priorities and perspectives
Chapter 7. [Step 6] Strengthening the Evaluation Design and the Validity of Conclusions
Chapter 8. [Step 7] Making it Useful: Helping clients and other stakeholders use the evaluation findings and recommendations
Part 2. A Review of Evaluation Methods and Approaches and their Application in Real World Evaluation: For those who would like to dig deeper on particular evaluation topics
Chapter 9. Ensuring competent and ethical practice in the conduct of the evaluation
Chapter 10. Theory-based evaluation
Chapter 11. Evaluation designs
Chapter 12. Quantitative evaluation approaches
Chapter 13. Qualitative evaluation approaches
Chapter 14. Mixed method evaluations
Chapter 15. Sampling
Chapter 16. Evaluating complex, multi-component development interventions
Part 3. Organizing and Managing Evaluations and Strengthening Evaluation Capacity: For readers involved with the funding and management of evaluations
Chapter 17. Organizing and managing evaluations
Chapter 18. Strengthening evaluation capacity
Chapter 19. Conclusions and challenges and the road ahead