Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests / Edition 1

Paperback (Print)
Buy New
Buy New from BN.com
$58.80
Used and New from Other Sellers
Used and New from Other Sellers
from $39.40
Usually ships in 1-2 business days
(Save 43%)
Other sellers (Paperback)
  • All (7) from $39.40   
  • New (5) from $50.20   
  • Used (2) from $39.40   

Overview

In Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests, authors Gregory J. Cizek and Michael B. Bunch provide the only "go-to" reference on how to set standards on tests in education, licensure, and certification. This book is comprehensive in scope, practical in nature, and definitive in terms of cataloguing the essential conceptual and procedural fundamentals of setting performance standards.

Key Features:

  • Offers a detailed description of procedures: In a practical, straightforward, and virtually jargon-free style, this book begins with important foundational and conceptual information about standard setting followed by step-by-step procedures for all currently used methods, including Angoff, Item Mapping, Analystic Judgment, etc. In addition, a full chapter is devoted to future issues and research in standard setting.
  • Enables readers to design, implement, and evaluate a standard setting study: Using this book, readers will be able to: select an appropriate standard setting method; schedule and plan implementation; identify and train qualified standard-setting participants; conduct the standard setting meeting; calculate one or more cut scores; and evaluate the success of the procedure they have implemented.
  • Allows readers to adapt written materials to their situation: Examples are provided in widely available formats such as Excel, SPSS, .pdf and Word files so readers do not have to purchase specialized software or learn to use unfamiliar packages. The book includes downloadable forms, sample correspondence, checklists, data sets, and analytical software for performing calculations required for various standard setting procedures.

Intended Audience:

This is an excellent supplementary textbook for graduate courses such as Applied Measurement in Education; Psychometric Theory; Advanced Educational Measurement; Special Topics in Measurement and Evaluation; Seminar in Educational and Psychological Measurement in the departments of Educational Psychology, Measurement and Evaluation, Psychology, and others.

Read More Show Less

Editorial Reviews

Psyc CRITIQUES: Contemporary Psychology
A well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites.
PsycCritiques
A well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites.
Psyc Critiques
A well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites.
Journal of Educational Measurement
It is abundantly apparent that the authors know standard setting and themselves have implemented numerous standard-setting procedures using different methods for different types of assessment programs.... Although the book is very practical, someone who has a fair amount of knowledge in standard setting will not feel shortchanged. There is a richness in the discussions of the methods that engages the reader, such as the different modifications of a method and the situational challenge that warranted each modification. Also, when a specific implementation of a method is discussed in detail, the authors are very conscientious in discussing the impetus for the adaptations. The historical perspective presented for some of the methods is quite intriguing.
Read More Show Less

Product Details

  • ISBN-13: 9781412916837
  • Publisher: SAGE Publications
  • Publication date: 11/28/2006
  • Edition description: New Edition
  • Edition number: 1
  • Pages: 368
  • Product dimensions: 6.00 (w) x 8.90 (h) x 0.90 (d)

Meet the Author

Gregory J. Cizek is Professor of Educational Measurement at the University of North Carolina, Chapel Hill. His background in the field of educa­tional assessment includes five years as a manager of licensure and certification testing programs for American College Testing (ACT) in Iowa City, Iowa, and 15 years of teaching experience at the col­lege level, where his teaching assignments have consisted primarily of graduate courses in educational testing, research methods, and statistics. He is the author of over 200 books, chapters, articles, conference papers, and reports. His books include Handbook of Educational Policy (Academic Press, 1998); Cheating on Tests: How to Do It, Detect It, and Prevent It (Lawrence Erlbaum, 1999); Setting Performance Standards: Concepts, Methods, and Perspectives (Lawrence Erlbaum, 2001); and Detecting and Preventing Classroom Cheating (Corwin Press, 2003). Dr. Cizek has served as an elected member and vice president of a local school board in Ohio, and he currently works with several states, organizations, and the U.S. Department of Education on tech­nical and policy issues related to large-scale standards-based testing programs for students in grades K–12. He began his career as an ele­mentary school teacher in Michigan, where he taught second and fourth grades.

Michael B. Bunch is Vice-President of Measurement Incorporated, a test development and scoring company serving the statewide assessment community. Dr. Bunch joined MI in 1982 and has built the research and development division of that organization into a team of more than 60 project directors, psychometricians, editors, content specialists, and support staff. Prior to joining MI, Dr. Bunch was a senior professional with NTS Research Corporation, where he provided ESEA Title I evaluation technical assistance to state education agencies. From 1976 to 1978, he was a research psychologist with the American College Testing Program (ACT). Dr. Bunch has authored dozens of professional and technical reports on a variety of topics—many on standard setting—and has presented his work at national conferences, including annual meetings of the American Educational Research Association (AERA), National Council on Measurement in Education (NCME), American Psychological Association (APA), Council of Chief State School Officers (CCSSO), Education Commission of the States (ECS) and Adult Education Research Conference (AERC). He received his Ph.D. in psychological measurement from the University of Georgia.

Read More Show Less

Table of Contents


Preface     xi
Fundamentals of Standard Setting     1
Contemporary Standard Setting: An Enduring Need     5
The Need to Make Decisions     6
The Benefits of Standard Setting     8
General Approaches to Standard Setting     9
Current Contexts for Standard Setting     11
What Is Standard Setting?     13
Kinds of Standards     14
Definitions of Standard Setting     14
Policy Issues and Standard Setting     19
Scoring Models     20
Research on Standard Setting     22
Rounding     23
Classification Errors     25
Item Scoring Criteria and Total-Test Performance Standards     29
Conclusions     33
Common Elements in Setting Performance Standards     35
Purpose     36
Choosing a Standard-Setting Method     41
Performance Level Labels     44
Performance Level Descriptions     46
Key Conceptualizations     48
Selecting and Training Standard-Setting Participants     49
Providing Feedback to Participants     53
Normative Feedback     54
Reality Feedback     55
Impact Feedback     56
Professional Guidelines for Standard Setting     57
Evaluating Standard Setting     59
Conclusions and a Foreword     63
Standard-Setting Methods     65
The Nedelsky Method     69
Procedures for the Nedelsky Method     70
Alternative Procedures and Limitations     71
The Ebel Method     75
Procedures for the Ebel Method     75
Alternative Procedures and Limitations     78
The Angoff Method and Angoff Variations     81
Procedures for the Angoff Method     82
Procedures for Angoff Variations     87
The Extended Angoff Method     87
The Yes/No Method     88
Alternative Procedures and Limitations     92
The Direct Consensus Method     97
Procedures for the Direct Consensus Method     98
Alternative Procedures and Limitations     102
The Contrasting Groups and Borderline Group Methods     105
Procedures for the Contrasting Groups Method     106
An Example Using the Contrasting Groups Method     108
The Borderline Group Method     112
Alternative Procedures and Limitations     113
The Body of Work and Other Holistic Methods     117
The Judgmental Policy Capturing Method     117
The Dominant Profile Method     120
The Analytical Judgment Method     121
Summary Analysis of Three Holistic Methods     122
Overview of the Body of Work Method     123
Procedures for the Body of Work Method     124
Training     125
Rangefinding     125
Pinpointing     129
Calculating Cut Scores     129
Interpreting the Logistic Regression Output     135
An Application of the Body of Work Method     138
Selecting Work Samples     139
Training Participants     140
Rangefinding     140
Alternative Procedures and Limitations     148
The Bookmark Method     155
Overview of the Bookmark Method     157
The Ordered Item Booklet     160
The Response Probability (RP) Value     162
Response Probabilities and Ordered Item Booklet Assembly-Rasch Model     162
Response Probabilities and Ordered Item Booklet Assembly-2PL Model     167
Directions to Bookmark Participants     172
Calculating Bookmark Cut Scores     176
An Implementation of the Bookmark Procedure     177
Training     177
Introducing the Ordered Item Booklet     179
Round One of a Bookmark Procedure     180
Obtaining Preliminary Bookmark Cut Scores     181
A Caveat and Caution Concerning Bookmark Cut Scores     184
Round One Feedback to Participants     185
Round Two of a Bookmark Procedure     186
Round Three of a Bookmark Procedure     187
Alternative Procedures and Limitations     189
The Item-Descriptor Matching Method     193
Procedures for the IDM Method     194
Alternative Procedures and Limitations     202
The Hofstee and Beuk Methods     207
The Hofstee Method     209
Procedures for Implementing the Hofstee Method     209
The Beuk Method     212
Procedures for Implementing the Beuk Method     212
Alternative Procedures and Limitations     214
Challenges and Future Directions in Standard Setting     217
Scheduling Standard-Setting Activities     219
Scheduling Standard Setting for Educational Assessments     219
Overall Plan     222
Participants     225
Materials      231
Final Preparations     234
At the Standard-Setting Site and Following Up     236
Scheduling Standard Setting for Credentialing Programs     237
Overall Plan     238
Participants     242
Materials     243
Final Preparations     244
At the Standard-Setting Site and Following Up     244
Conclusions and Recommendations     246
Vertically-Moderated Standard Setting     249
The Interrelated Challenges     250
A Brief History of Vertically-Moderated Standard Setting     253
What Is VMSS?     254
Approaches to VMSS     256
Applications of VMSS     257
An Illustration of VMSS Procedures     262
The Assessment Context     262
Preparing to Implement a VMSS Approach     263
Training VMSS Participants     264
Facilitating the VMSS Standard-Setting Meeting     265
Vertical Articulation of Cut Scores     266
Final Review, Adoption, and Conclusions     270
Alternative Procedures and Limitations     271
Standard Setting on Alternate Assessments     275
The Unique Challenges of Alternate Assessment     276
Necessary Conditions for Alternate Assessment Systems     278
A Generalized Holistic Method     283
Overview of an Application of the GH Method     285
Procedures for Implementing the GH Method     286
Conclusions and Discussion Regarding the GH Method     292
Alternative Procedures and Limitations     293
Special Topics and Next Steps     297
Rounding     298
Methods of Adjusting Cut Scores     299
Deciding How to Incorporate Uncertainty     302
Generalizability of Standards     306
Decision Consistency and Decision Accuracy     307
A Demonstration of Computing Decision Consistency and Decision Accuracy on Complex Tests     313
Other Decision Consistency Procedures     315
Summary and Future Directions     317
Using Multiple Methods of Standard Setting     319
Improving Participant Training     320
References     323
Glossary     333
Author Index     341
Subject Index     345
About the Authors     351
Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star

(0)

4 Star

(0)

3 Star

(0)

2 Star

(0)

1 Star

(0)

Your Rating:

Your Name: Create a Pen Name or

Barnes & Noble.com Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & Noble.com that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & Noble.com does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at BN.com or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation

Reminder:

  • - By submitting a review, you grant to Barnes & Noble.com and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Noble.com Terms of Use.
  • - Barnes & Noble.com reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & Noble.com also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on BN.com. It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

 
Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)