Measurement Demystified: Creating Your L&D Measurement, Analytics, and Reporting Strategy

by David Vance, Peggy Parskey

Overview

Your Groundbreaking Framework for Measurement and Reporting

Most people find measurement, analytics, and reporting daunting—and L&D professionals are no different. As these practices have become critically important for organizations’ efforts to improve performance, talent development professionals have often been slow to embrace them for many reasons, including the seeming complexity and challenge of the practices. Few organizations have a well-thought-out measurement and reporting strategy, and there are often scant resources, limited time, and imperfect data to work with when organizations do attempt to create one.

Measurement Demystified: Creating Your L&D Measurement, Analytics, and Reporting Strategy is a much-needed and welcomed resource that breaks new ground with a framework to simplify the discussion of measurement, analytics, and reporting as it relates to L&D and talent development practitioners. This book helps practitioners select and use the right measures for the right reasons; select, create, and use the right types of reports; and create a comprehensive measurement and reporting strategy.

Recognizing the angst and reluctance people often show in these areas, authors and experts David Vance and Peggy Parskey break down the practices and processes by providing a common language and an easy-to-use structure. They describe five types of reports, four broad reasons to measure, and three categories of measures. Their method works for large and small organizations, even if yours is an L&D staff of one or two. The guidance remains the same: Start small and grow.

Measurement Demystified is a great first book for talent development professionals with no prior knowledge of or experience with measurement and a valuable resource for measurement experts. Those adept at lower levels of training evaluation will grow their knowledge base and capabilities, while measurement experts will discover shortcuts and nuggets of information to enhance their practices. A more comprehensive treatment of these important topics will not be found elsewhere.

Product Details

ISBN-13:	9781950496891
Publisher:	Association for Talent Development
Publication date:	11/17/2020
Pages:	432
Sales rank:	214,265
Product dimensions:	10.00(w) x 7.00(h) x 0.89(d)

About the Author

David Vance is the executive director of the Center for Talent Reporting. He is the former president of Caterpillar University, which he founded in 2001, until his retirement in 2007. Prior to this position, Dave was chief economist and head of the Business Intelligence Group at Caterpillar Inc. Dave received a bachelor of science degree from M.I.T., a master of science degree in business administration from Indiana University (South Bend), and a Ph.D. in economics from the University of Notre Dame. He was named 2006 Chief Learning Officer (CLO) of the Year by Chief Learning Officer magazine. He also was named 2004 Corporate University Leader of the Year by the International Quality and Productivity Center in their annual CUBIC (Corporate University Best in Class) Awards. Caterpillar was ranked number one in the 2005 ASTD Best Awards and was named Best Overall Corporate University in 2004 by both Corporate University Xchange and the International Quality and Productivity Center. Dave is a frequent speaker at learning conferences and association meetings. He conducts workshops on measurement and reporting and running learning like a business. He also organizes and hosts the Center for Talent Reporting’s annual conference. Dave teaches in the Ph.D. programs of Bellevue University and the University of Southern Mississippi as well as in the executive education program at George Mason University. He is the author of The Business of Learning: How to Manage Corporate Training to Improve Your Bottom Line. Peggy Parskey is the executive director of the Center for Talent Reporting. Peggy owns her own consulting firm, Parskey Consulting, enabling her clients to successfully implement strategic change initiatives that improve organizational and individual performance. Peggy has a deep background in performance measurement and leverages her expertise in management of change and organizational design to ensure sustainable capability. Peggy is certified in management of change methodologies both at the organizational and individual performer levels. She holds a bachelor of science degree in mathematics from Simmons College and two master’s degrees from the University of Chicago in statistics and business administration. Peggy is also a part-time principal consultant at Explorance. In this role, Peggy consults with organizations to develop talent measurement strategies, integrate measurement into talent processes, develop action-oriented reports, scorecards and dashboard for clients, and conduct impact studies to demonstrate the linkage between talent programs and business outcomes. Prior to working with the Center for Talent Reporting and Explorance, Peggy was employed at Hewlett-Packard, where she was responsible for global learning processes focused on creating best-in-class learning methodologies as well as enterprise wide evaluation for the L&D function. She has published several articles on measurement, chapters in two books, and is the co-author of the second edition of Learning Analytics, Using Talent Data to Improve Business Outcomes.

Table of Contents

List of Figures vii

List of Tables ix

Foreword xvii

Preface xxiii

Introduction 1

Part I Foundations of Measurement

1 The Talent Development Reporting Principles (TDRp) Framework 11

Reasons for Measuring 11

A Framework for Measurement 12

Measurement Maturity 14

TDRp Measures 18

TDRp Reports 20

The Complete TDRp Framework 23

Integration of the TDRp Framework With Other Standard-Setting Efforts 25

Conclusion 29

2 Introduction to Basic Measures 31

The Consequences of Focusing on a Single Type of Measure 31

The Solution: Be Purposeful About What You Measure 37

Conclusion 39

Part II Measurement

3 Efficiency Measures 43

Formal Learning Measures 46

Informal Learning Measures 65

Conclusion 71

4 Effectiveness Measures 75

The Five Levels of Effectiveness Measures 75

Level 1: Reaction 79

Level 2: Learning 85

Level 3: Behavior and Application 88

Level 4: Results and Impact 95

Level 5: ROI 106

Conclusion 113

5 Outcome Measures 115

The Importance of Outcome Measures 115

Most Common Outcome Measures 116

Types of Outcome Measures 117

Application Rate as a Leading Indicator 118

Flexibility in Creating Outcome Measures 119

The Special Case of Competency Measures 119

Conclusion 121

6 Creating a Measurement Strategy 123

Why a Measurement and Reporting Strategy Is Important 123

Steps to Create a Measurement Strategy 124

Elements of a Measurement and Reporting Strategy 126

Conclusion 141

7 Selecting Measures 143

Measures for Programs 145

Measures for General Studies Courses 152

Department Initiatives 154

Other Measures the CLO May Request 159

Conclusion 160

Part III Reporting

8 Reporting 165

Types of Reports and Data Displays 165

Reporting to Inform 166

Reporting to Monitor 170

Reporting to Evaluate and Analyze 177

Reporting to Manage 180

Conclusion 183

9 The Three Management Reports 185

Running Learning Like a Business 185

The TDRp Management Reporting Framework 197

Conclusion 228

10 Creating Your Reporting Strategy 231

Elements of a Measurement and Reporting Strategy 231

Report Types and Formats 232

Plans to Generate and Distribute the Reports 239

Resources Required for Reporting 243

Conclusion 244

11 Creating Plans 247

Creating Plans for Outcome Measures 249

Creating Plans for Efficiency and Effectiveness Measures 264

Conclusion 270

12 Reporting Year-to-Date Results and Making Forecasts 273

Reporting YTD Results 274

Making Forecasts 284

Conclusion 296

Part IV Putting It All Together

13 Implementing Measurement and Reporting 299

L&D's Opportunity 301

What's Changing? 302

Managing the Change 308

Tools and Templates 313

Conclusion 321

14 Pulling It All Together 325

Key Takeaways 325

Opportunities 329

Closing Thoughts 331

Appendixes

A History of the TDRp Approach 335

B Sample Roles and Responsibilities Agreement 337

C Sample Measurement and Reporting Strategy 341

D Sample TDRp Implementation Plan 369

E Glossary 375

F Resources on Survey Design 395

References 397

Index 399

About the Authors 407

Editorial Reviews

Dave Vance and Peggy Parskey have delivered a practical, flexible approach to data analytics and reporting—workable for organizations of any size. If you are an L&D professional, all the more tentative to start or proceed in these times of limited resources and uncertainty, let the authors show you how measurement and analytics can help you to create a compelling case for any business challenge.

Kimo Kippen

Measurement Demystified provides a proven methodology I have used to drive business buy-in and award-winning results. This is must-read for talent leaders of the future."

Joshua Craver

Measurement Demystified is a must read for anyone in talent development. It provides a practical and comprehensive framework for measuring and reporting on industry accepted L&D metrics. This is the go-to handbook we’ve all been waiting for!

Brenda Sugrue

At last, a truly comprehensive guide to measuring, evaluating, and reporting the outcomes of human capital development programs. Filled with clear definitions, helpful diagrams, and practical advice, this book will be a boon to learning professionals everywhere.

Roy Pollock