Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness

by Huey Chen

Overview

Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness introduces students to the real world of evaluation, focusing on issues that arise in professional practice. Author Huey T. Chen concentrates on the steps vital to program evaluation: systematically identifying stakeholder needs, selecting evaluation options best suited to particular needs and reconciling any necessary tradeoffs, and turning the decisions into action. Using illustrative examples from the field, the book details the methods and procedures involved in analyzing and identifying stakeholder expectations and selecting an evaluation strategy. The principles and strategies presented are reinforced with theoretical justification. This provides students and professional evaluators the knowledge and flexibility to design suitable evaluations and follow them through from start to finish.

Product Details

ISBN-13:	9780761902324
Publisher:	SAGE Publications
Publication date:	10/28/2004
Pages:	304
Product dimensions:	5.90(w) x 9.10(h) x 0.90(d)

About the Author

Huey T. Chen has been a Professor at the University of Alabama at Birmingham since 2002. He was born and raised in Taiwan. Dr. Chen worked at the University of Akron until 1997, when he joined the Center for Disease Control and Prevention (CDC) as the chief of an evaluation branch. Dr. Chen had taken a leadership role in designing and implementing a national evaluation system for assessing the CDC funded HIV prevention programs, which are based in health departments and community organizations. Dr. Chen has contributed to the development of evaluation theory and methodology, especially in the areas of program theory and theory-driven evaluations. His book, Theory-Driven Evaluations, has been recognized as one of the landmarks in program evaluation. In 1998 he received the Senior Biomedical Research Service Award from the CDC. He is also the 1993 recipient of the Paul F. Lazarsfeld Award for contributions to Evaluation Theory from the American Evaluation Association.

Table of Contents

	Preface	ix
	Overview of the Book	xi
Part I	Introduction	1
1	Fundamentals for Practicing Program Evaluation	3
	The Nature and Characteristics of Programs	3
	The Nature and Characteristics of Effective Evaluation	6
	The Universalist View of the Practice of Evaluation (and Its Limitations)	10
	The Contingency View and the Program Theory Perspective	11
	Audiences and Uses of This Book	13
2	A Conceptual Framework of Program Theory for Practitioners	15
	The Program Theory Conceptual Framework	16
	Components of the Change Model	20
	Components of the Action Model	23
	Relationships Among Components of the Conceptual Framework of Program Theory	29
	Applied Program Theory: An Example	32
	Relationships Between "Program Theory" and "Logic Model"	34
	Some Advantages of Using the Program Theory Conceptual Framework	36
	Scientific Theory Versus Stakeholder-Implicit Theory as a Basis for Programs	38
	Applications of the Framework Presented in the Book	42
3	A Practical Evaluation Taxonomy: Selecting the Evaluation Approach That Works	45
	The Art of Fishing? The Art of Program Evaluation?	45
	The Evaluation Taxonomy as a Source for the Art Aspect of Program Evaluation	47
	A Practical Taxonomy of Program Evaluation Means and Ends	49
	Steps to Take in Applying the Practical Taxonomy	58
	Evaluation Ranging Across Several Program Stages	61
	Helping Stakeholders Gear Up (or Clear Up) Their Program Theory	63
	Details of the Work: Designing an Evaluation	68
	Dynamics of Evaluation Entries Into Program Stages and How to Apply This Book to Conducting Evaluation	68
Part II	Program Evaluation to Help Stakeholders Plan Intervention Programs	71
4	Assisting Stakeholders as They Formulate Program Rationales	73
	The Concept of a Program Rationale and Its Purposes	73
	Strategies for Articulating Program Rationales	76
	Background Information Provision Strategy and Approaches	76
	The Conceptualization Facilitation Approach-A Part of the Development Facilitation Strategy	79
	The Relevancy Testing Approach-A Part of the Troubleshooting Strategy	88
5	How Evaluators Assist Stakeholders in Developing Program Plans	93
	The Action Model Framework and the Program Plan	93
	Strategies for Developing Program Plans	96
	The Formative Research Approach, Under Background Information Provision Strategy	97
	The Conceptualization Facilitation Approach, Under Development Facilitation Strategy	98
	Application of the Conceptualization Facilitation Strategy	111
	The Pilot-Testing Approach	119
	The Commentary or Advisory Approach	125
	Summary	127
Part III	Evaluating Implementation	129
6	Development-Oriented Evaluation Tailored for the Initial Implementation	131
	The Formative Evaluation Approach, Under the Troubleshooting Strategy	132
	The Program Review/Development Meeting, Under the Troubleshooting Strategy	144
	Bilateral Empowerment Evaluation, Under the Development Partnership Strategy	150
7	Assessing Implementation in the Mature Implementation Stage	155
	Development-Oriented and Assessment-Oriented Approaches	155
	Application of Development-Oriented Process Evaluation in the Mature Implementation Stage	156
	Assessment-Oriented Process Evaluation	159
	Issues in Applying Assessment-Oriented Process Evaluation in the Mature Implementation Stage	161
	The Fidelity Evaluation Approach, Under the Performance Assessment Strategy	165
	Enlightenment Assessment Strategy and Theory-Driven Process Evaluation	173
Part IV	Program Monitoring and Outcome Evaluation	179
8	Monitoring the Progress of a Program	181
	What Is Program Monitoring?	181
	Process Monitoring	182
	Outcome Monitoring	184
	Program Monitoring Systems Within Organizations	189
9	Evaluating Outcomes: Efficacy Evaluation Versus Effectiveness Evaluation	195
	Guidelines for Conducting Outcome Evaluation	195
	Evaluability Assessment, Under the Development Facilitation Strategy	197
	Nonlinear Transitions Between Program Stages	199
	Efficacy Evaluation or Effectiveness Evaluation?	199
	Efficacy Evaluation's Conceptual Framework and Methodology	201
	Effectiveness Evaluation in the Current State of the Art	205
	Reconsidering the Efficacy-to-Effectiveness Paradigm	206
	Effectiveness Evaluation Adopts Efficacy Evaluation Methodology and Its Limitations	209
	The Conceptual Framework of Effectiveness Evaluation	212
	Designing and Conducting Effectiveness Evaluation	214
	Addressing Differential Implementation of Interventions	225
	Anticipating Intrusions and Preparing for Problems	227
	Beyond the Basic Effectiveness and Efficacy Evaluations	227
10	Theory-Driven Outcome Evaluation	231
	A Reply to Criticisms of Theory-Driven Outcome Evaluation	233
	Issues on the Difficulty of Clarifying Stakeholders' Implicit Theory	235
	Issues on Program Theory Consensus Among Stakeholders	238
	Guidelines for Conducting Theory-Driven Outcome Evaluation	238
	Types of Theory-Driven Outcome Evaluation	240
	The Intervening Mechanism Evaluation Approach	241
	The Moderating Mechanism Evaluation Approach	250
	The Integrative Process/Outcome Evaluation Approach	256
	Theory-Driven Outcome Evaluation and Unintended Effects	262
11	Looking Forward	267
	Evaluators' Plural Roles	268
	Politics of Program Evaluation	269
	Evaluation Practitioners' Contributions to the Development of Evaluation Theories and Methodology	269
	Narrowing the Gaps Between Academic and Practical Communities	270
	References	273
	Index	281
	About the Author	292

Editorial Reviews

Practical Program Evaluation presents a useful and usable model for deciding which kind of evaluation activities to undertake in different circumstances. Chen identifies different kinds of evaluation activities that are appropriate for different stages of a program. He also shows how information needs inside and outside the program, as well as a set of possible trade-offs, should affect the choice of evaluation activities. Chen provides a new and important framework for thinking about these fundamental issues. That is no small accomplishment. Like Chen's previous book on theory-driven evaluation, this book should have a major impact on evaluation theory and practice.

Melvin Mark