- Shopping Bag ( 0 items )
How to Measure Training Results presents practical tools for collecting and measuring six types of data critical to an overall evaluatin of training. This timely resource:
The Need for and Benefits of Measurement and Evaluation of Training Outcomes
WHY MEASUREMENT AND EVALUATION ARE NECESSARY
Much has been written about the need for training and performance-improvement professionals to become more accountable and to measure their contributions. And the organization funds training at the expense of other organizational needs, and the results influenced by training can be elusive without a focused evaluation effort to address the outcomes. Just as learning initiatives must include the various stakeholders, so too must the evaluation effort include the stakeholders of the organization. In essence, the training function must be a business partner in the organization in order to successfully deliver its products. Most observers of the field have indicated that for performance practitioners to become true business partners, three things must be in place.
1. Training and performance-improvement initiatives must be integrated into the overall strategic and operational framework of the organization. They cannot be isolated, event-based activities, unrelated to the mainstream functions of the business.
2. There must be a comprehensive measurement and evaluation process to capture the contributions of human resource development and establish accountability. The process must be comprehensive, yet practical, and feasible as a routine function in the organization.
3. Partnership relationships must be established with key operating managers.
These key clients are crucial to the overall success of the training function. Most training executives believe their function is now an important part of the business strategy. During the 1990s, and continuing into the twenty-first century, training and performance improvement have become a mainstream function in many organizations. The training executives of these organizations emphasize the importance of successfully establishing partnerships with key management and report that tremendous strides have been made in working with managers to build the relationships that are necessary. They report fair progress in the achievement of integrating training into the overall strategic and operational framework of the organization. However, they indicate that there has not been progress on the second condition: a comprehensive measurement and evaluation process—at least not to the extent needed in most organizations. This book is devoted to presenting the principles and tools necessary to allow practitioners to implement a comprehensive measurement and evaluation process to improve results in their organization. The installation of a comprehensive measurement and evaluation process will quite naturally address the other two items as well. The comprehensive measurement of training will provide for a closer link to the organization's strategic goals and initiatives. Measurement will also allow line managers to see the results as well as the potential from training efforts, and this will lend itself to stronger partnerships.
Measurement will continue to be necessary as long as the drivers for accountability exist. Some of the current drivers for accountability are operating managers' concern with bottom line, competition for funds and resources, accountability trend with all functions, top-management interest in ROI, and continuing increases in program costs. In the final analysis, the real issues behind accountability are the external forces of competition. In the business sector it is the competitive nature of the world marketplace. In government and nonprofit organizations, it is the competition for funds and resources to achieve the primary mission.
A FRAMEWORK FOR EVALUATION WITH SIX TYPES OF MEASURES
Measurement and evaluation are useful tools to help internalize the results- based culture and to track progress. When looking for evidence of accountability in training, the question of what to measure and what data to review is at the heart of the issue.
Applying the framework presented in this chapter, along with the ROI (return on investment) process, involves five types of data (associated with five levels of measurement) and a sixth type of data represented by intangible benefits. These can be used to measure training and educational programs, performance- improvement programs, organizational change initiatives, human resource programs, technology initiatives, and organization development initiatives. (For consistency and brevity, we use the term "training programs" throughout most of this book.)
The fifth level in this framework is added to the four levels of evaluation developed for the training profession almost 40 years ago by Donald Kirkpatrick. The concept of different levels of evaluation is helpful in understanding how the return on investment is calculated.
Table 1.1 shows the modified version of the five-level framework as well as the intangible dimension.
The six types of data that are the focal point of this book are all useful in their own ways and for specific purposes. A chapter in this book is dedicated to each of the six types of data. Each chapter presents the merits of the specific type of data and how it is used.
When planning an evaluation strategy for a specific program, an early determination must be made regarding the level of evaluation to be used. This decision is always presented to interested stakeholders for their input and guidance. For example, if you have decided to evaluate a specific program (or a stakeholder has asked for an evaluation), you should first decide the highest level of evaluation that is appropriate. This should guide you as to the purpose of the evaluation study. You should then ascertain what data is acceptable to the various stakeholders and what interests and expectations they have for each of the five levels. After an appropriate discussion about possible intangibles, you should seek their opinions and expectations about the inclusion of intangible data.
The five levels
The levels represent the first five of the six measures (key indicators) discussed in this book. At Level 1, Reaction, Satisfaction, and Planned Actions, participants' reactions to the training are measured, along with their input on a variety of issues related to training design and delivery. Most training programs are evaluated at Level 1, usually by means of generic questionnaires or surveys.
Although this level of evaluation is important as a measure of customer satisfaction, a favorable reaction does not ensure that participants have learned the desired facts, skills, etc., will be able to implement them on the job, and/or will be supported in implementing them on the job. An element that adds value to a Level-1 evaluation is to ask participants how they plan to apply what they have learned.
At Level 2, Learning, measurements focus on what the participants learned during the training. This evaluation is helpful in determining whether participants have absorbed new knowledge and skills and know how to use them as a result of the training. This is a measure of the success of the training program. However, a positive measure at this level is no guarantee that the training will be successfully applied in the work setting.
At Level 3, Application and Implementation, a variety of follow-up methods are used to determine whether participants actually apply what they have learned from the training to their work settings. The frequency and effectiveness of their use of new skills are important measures at Level 3. Although Level-3 evaluation is important in determining the application of the training, it still does not guarantee that there will be a positive impact on the
Excerpted from How to Measure Training Results by Jack Phillips. Copyright © 2000 by The McGraw-Hill Companies. Excerpted by permission of The McGraw-Hill Companies, Inc..
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
Chapter 1 The Need for and Benefits of Measurement and Evaluation of
Chapter 2 The ROI Model and Process
Chapter 3 Step 1. Develop Training Objectives: The Basis for Measurement
Chapter 4 Step 2. Develop Evaluation Plans and Baseline Data
Chapter 5 Step 3. Collect Data During Training (Levels 1 and 2)
Chapter 6 Step 4. Collect Data After Training (Levels 3 and 4)
Chapter 7 Step 5. Isolate the Effects of Training
Chapter 8 Step 6. Convert Data to Monetary Values
Chapter 9 Step 7. Identify the Costs of Training
Chapter 10 Step 8. Calculate the Return on Investment (Level 5)
Chapter 11 Step 9. Identify Intangible Benefits
Chapter 12 Step 10. Generate an Impact Study
Chapter 13 Fast and Easy Approaches to Measuring Training Outcomes
Chapter 14 Gaining Management Support and Implementing the Process
Posted December 18, 2008
Authors Jack J. Phillips and Ron Drew Stone have created an easy-to-understand method for measuring the real benefits of training. They outline a process you can use not only to understand the real impact of your training programs, but also to explain the benefits to top management in terms of actual dollars. Their guide takes you step-by-step through six levels of evaluation, from the early planning stages through implementation, right up to an impact study you can use on an ongoing basis to manage training evaluation. getAbstract thinks this guide offers very useful information for training managers. It provides concrete, step-by-step formulas and practical examples on how to set training goals, evaluate training at each level, convert training data to a compelling ROI model and measure the real benefits of your training programs.Was this review helpful? Yes NoThank you for your feedback. Report this reviewThank you, this review has been flagged.