Program Evaluation: Alternative Approaches and Practical Guidelines / Edition 4

by Jody Fitzpatrick

Overview

The most comprehensive text on the market, Program Evaluation: Alternative Approaches and Practical Guidelines, Third Edition provides an overview of a wide variety of approaches to evaluation and extensive practical guidelines for carrying out evaluation studies successfully. This text helps both students and professionals who are new to evaluation to understand how the field has evolved, what different approaches an evaluator can take in conducting evaluations, and how to plan and conduct an evaluation. The text makes extensive use of checklists, examples, and a comprehensive case study. Finally, throughout the book, students are introduced to current trends and controversial issues in evaluation and ways to conduct evaluations in an ethical and professional manner.

Product Details

ISBN-13:	2900205579357
Publication date:	09/27/2010
Pages:	568
Product dimensions:	6.50(w) x 1.50(h) x 9.50(d)

About the Author

Jody Fitzpatrick is an emeritus faculty member in public administration at the University of Colorado Denver, where she taught research methods and evaluation. She has conducted evaluations in many schools and human service settings and written extensively about successful evaluation practice. She served on the board and as president of the American Evaluation Association (AEA) and on the editorial boards of the American Journal of Evaluation and New Directions for Evaluation. She also chaired AEA's Teaching of Evaluation topical interest group and won a teaching award at the University of Colorado Denver. In her book, Evaluation in Action: Interviews with Expert Evaluators, she used interviews with expert evaluators to talk about the decisions that evaluators face as they plan and conduct evaluations and the factors that influence their choices. She evaluated the changing roles of counselors in middle schools and high schools and a program to help immigrant middle-school girls to achieve and stay in school. Her international work includes research on evaluation in Spain and elsewhere in Europe, and she has spoken on evaluation issues to policymakers and evaluators in France, Spain, Denmark, Mexico, and Chile.

James Sanders is professor emeritus of educational studies and former associate director of The Evaluation Center at Western Michigan University, where he taught, published, consulted, and conducted evaluations. A graduate of Bucknell University and the University of Colorado, he served on the board and as president of the American Evaluation Association (AEA). He chaired the steering committee that created the Evaluation Network, a predecessor to AEA. His publications include books on school, student, and program evaluation. He has worked extensively with schools, foundations, and government and nonprofit agencies to develop their evaluation practices. As chair of the Joint Committee on Standards for Educational Evaluation, he led the development of the second edition of the Program Evaluation Standards. He was also involved in developing the concepts of applied performance testing for student assessments, cluster evaluation for program evaluations by foundations and government agencies, and mainstreaming evaluation for organizational development. His international work in evaluation concentrated in Canada, Europe, and Latin America. He received distinguished service awards from Western Michigan University, where he helped establish a Ph.D. program in evaluation, and from the Michigan Association for Evaluation.

Blaine Worthen is psychology professor emeritus at Utah State University, where he founded and directed the evaluation methodology Ph.D. program and the Western Institute for Research and Evaluation, conducting more than 350 evaluations for local and national clients in the United States and Canada. He received his Ph.D. from The Ohio State University. He is a former editor of Evaluation Practice and founding editor of the American Journal of Evaluation. He served on the American Evaluation Association (AEA) board of directors and received AEA's Alva and Gunnar Myrdal Evaluation Practice Award and the American Education Research Association's Best Evaluation Study Award. He taught university evaluation courses for three decades, managed federally mandated evaluations in 17 states, advised numerous government and private agencies, and has given more than 150 keynote addresses and evaluation workshops in the United States, England, Australia, Israel, Greece, Ecuador, and other countries. He has written extensively on evaluation, measurement and assessment and is the author of 135 articles and six books. His Phi Delta Kappan article, "Critical Issues That Will Determine the Future of Alternative Assessment," was distributed to 500 distinguished invitees at the White House’s Goals 2000 conference. He is recognized as a national and international leader in the field.

Lori Wingate is the executive director of The Evaluation Center at Western Michigan University (WMU), where she has worked since 1997. She has a Ph.D. in interdisciplinary evaluation from WMU and has been working in the field of program evaluation since 1993. From 2009 to 2019, she directed EvaluATE, the evaluation hub for the National Science Foundation's Advanced Technological Education program. From 2011 to 2019, she served as a subject matter expert in evaluation for the U.S. Centers for Disease Control and Prevention (CDC). She has led more than 75 webinars and workshops on evaluation in various contexts, including CDC University, American Evaluation Association Summer Evaluation Institute, and EvaluATE. She leads the Evaluation Checklist Project at WMU and has developed numerous resource materials to support evaluation practice, including checklists, templates, and guides. She has written book chapters on evaluating humanitarian response to emergencies, evaluation checklists, and metaevaluation. She was the book review section editor for the American Journal of Evaluation from 2005 to 2011 and has led a range of evaluation projects in the areas of STEM education, public health, and higher education.

Table of Contents

	Preface	xiii
Part 1	Introduction to Evaluation	1
1	Evaluation's Basic Purpose, Uses, and Conceptual Distinctions	3
	A Brief Definition of Evaluation	4
	Informal versus Formal Evaluation	8
	Distinguishing between Evaluation's Purposes and Evaluators' Roles and Activities	9
	Some Basic Types of Evaluation	16
	Evaluation's Importance--and Its Limitations	26
2	Origins and Current Trends in Modern Program Evaluation	30
	The History and Influence of Evaluation in Society	30
	Recent Trends Influencing Program Evaluation	44
Part 2	Alternative Approaches to Program Evaluation	53
3	Alternative Views of Evaluation	57
	Diverse Conceptions of Program Evaluation	58
	Origins of Alternative Views of Evaluation	59
	Themes among the Variations	67
	A Classification Schema for Evaluation Approaches	68
4	Objectives-Oriented Evaluation Approaches	71
	Developers of the Objectives-Oriented Evaluation Approach and Their Contributions	72
	How the Objectives-Oriented Evaluation Approach Has Been Used	80
	Strengths and Limitations of the Objectives-Oriented Evaluation Approach	82
5	Management-Oriented Evaluation Approaches	88
	Developers of the Management-Oriented Evaluation Approach and Their Contributions	89
	How the Management-Oriented Evaluation Approach Has Been Used	94
	Strengths and Limitations of the Management-Oriented Evaluation Approach	95
6	Consumer-Oriented Evaluation Approaches	100
	Developers of the Consumer-Oriented Evaluation Approach and Their Contributions	101
	How the Consumer-Oriented Evaluation Approach Has Been Used	104
	Strengths and Limitations of the Consumer-Oriented Evaluation Approach	108
7	Expertise-Oriented Evaluation Approaches	112
	Developers of the Expertise-Oriented Evaluation Approach and Their Contributions	114
	How the Expertise-Oriented Evaluation Approach Has Been Used	121
	Strengths and Limitations of the Expertise-Oriented Evaluation Approach	123
8	Participant-Oriented Evaluation Approaches	129
	Evolution of Participant-Oriented Evaluation Approaches	130
	Developers of the Participant-Oriented Evaluation Approach and Their Contributions	131
	How Participant-Oriented Evaluation Approaches Have Been Used	145
	Strengths and Limitations of Participant-Oriented Evaluation Approaches	146
9	Alternative Evaluation Approaches: A Summary and Comparative Analysis	152
	Cautions about the Alternative Evaluation Approaches	153
	Contributions of the Alternative Evaluation Approaches	158
	Comparative Analysis of Characteristics of Alternative Evaluation Approaches	159
	Eclectic Uses of the Alternative Evaluation Approaches	163
	Drawing Practical Implications from the Alternative Evaluation Approaches	165
Part 3	Practical Guidelines for Planning Evaluations	169
	Introduction of Case Study	170
10	Clarifying the Evaluation Request and Responsibilities	173
	Understanding the Reasons for Initiating the Evaluation	174
	Conditions under which Evaluation Studies Are Inappropriate	178
	Determining When an Evaluation Is Appropriate: Evaluability Assessment	182
	Using an Internal or External Evaluator	185
	Hiring an Evaluator	189
	How Different Evaluation Approaches Clarify the Evaluation Request and Responsibilities	192
11	Setting Boundaries and Analyzing the Evaluation Context	199
	Identifying Intended Audiences for an Evaluation	200
	Describing What Is to Be Evaluated: Setting the Boundaries	203
	Analyzing the Resources and Capabilities That Can Be Committed to the Evaluation	212
	Analyzing the Political Context for the Evaluation	216
	Variations Caused by the Evaluation Approach Used	217
	Determining Whether to Proceed with the Evaluation	219
12	Identifying and Selecting the Evaluation Questions and Criteria	232
	Identifying Appropriate Sources of Questions and Criteria: The Divergent Phase	234
	Selecting the Questions, Criteria, and Issues to Be Addressed: The Convergent Phase	246
	Remaining Flexible during the Evaluation: Allowing New Questions, Criteria, and Standards to Emerge	253
13	Planning How to Conduct the Evaluation	260
	Identifying Design and Data Collection Methods	262
	Specifying How the Evaluation Will Be Conducted: The Management Plan	275
	Establishing Evaluation Agreements and Contracts	285
Part 4	Practical Guidelines for Conducting and Using Evaluations	301
14	Collecting Evaluation Information: Design, Sampling, and Cost Choices	303
	Using Mixed Methods	304
	Designs for Collecting Causal and Descriptive Information	307
	Sampling	320
	Cost Analysis	324
15	Collecting Evaluation Information: Data Sources and Methods, Analysis, and Interpretation	334
	Common Sources and Methods for Collecting Information	335
	Planning and Organizing the Collection of Information	356
	Analysis of Data and Interpretation of Findings	358
16	Reporting and Using Evaluation Information	375
	Purposes of Evaluation Reports	376
	Important Factors in Planning Evaluation Reports	377
	Key Components of a Written Report	382
	Suggestions for Presenting Information in Written Reports	388
	Alternative Methods for Reporting: The Adversary Approach	394
	Human and Humane Considerations in Reporting Evaluation Findings	395
	Suggestions for Effective Oral Reporting	398
	A Checklist for Good Evaluation Reports	400
	How Evaluation Information Is Used	400
17	Dealing with Political, Ethical, and Interpersonal Aspects of Evaluation	411
	Establishing and Maintaining Good Communications among Evaluators and Stakeholders	412
	Understanding Potential Bias Resulting from the Evaluator's Personal Values and Interpersonal, Financial, and Organizational Relationships with Others	415
	Maintaining Ethical Standards: Considerations, Issues, and Responsibilities for Evaluators and Clients	423
	Political Pressures and Problems in Evaluation	432
18	Evaluating Evaluations	442
	The Concept and Evolution of Metaevaluation	443
	The Joint Committee's Standards for Program Evaluation	444
	Summary of the Program Evaluation Standards	445
	AEA Guiding Principles for Evaluators	449
	The Role of Metaevaluator	451
	Some General Guidelines for Conducting Metaevaluations	453
	A Need for More Metaevaluation	455
Part 5	Emerging and Future Settings for Program Evaluation	461
19	Conducting Multiple-Site Evaluation Studies	463
	Purposes and Characteristics of Multiple-Site Evaluations	464
	Multisite Evaluation (MSE)	466
	On-Site Evaluation at Multiple Sites	471
	Cluster Evaluation	475
	Other Approaches to Multiple-Site Evaluation	481
20	Conducting Evaluation of Organizations' Renewal and Training in Corporate and Nonprofit Settings	485
	Evaluation in the Nonprofit Sector	486
	Evaluating Corporate Training Programs	491
	Personnel Evaluation	495
	Other Methods of Organizational Assessment	497
21	The Future of Evaluation	507
	The Future of Evaluation	508
	Predictions concerning the Profession of Evaluation	508
	Predictions concerning the Practice of Evaluation	510
	A Vision for Evaluation	513
	Conclusion	513
	Suggested Readings	514
Appendix	Evaluation-Related Web Sites	515
	References	519
	Author Index	543
	Subject Index	551

Editorial Reviews

"This is a landmark text and it remains the definitive classic for training new evaluators. The authors have truly made an important contribution to the field...Nothing comes close to this text in terms of coverage, balance of theory and practicality, and usefulness in the field of evaluation." - Gary J. Skolits, University of Tennessee

"This is a great textbook. It is simply the most readable, most comprehensive introduction to program evaluation." - Ted Miller, University of Tennessee, Chattanooga

From the Publisher