Brain Korea 21 Phase II: A New Evaluation Model

Brain Korea 21 Phase II: A New Evaluation Model




The Brain Korea 21 Program (BK21), which seeks to make Korean research universities globally competitive and to produce more high-quality researchers in Korea, provides funding to graduate students and professors who belong to research groups at top universities. The authors develop quantitative and qualitative models to evaluate how well BK21 is fulfilling its goals and make suggestions for further stimulating Korean university research.

Product Details

ISBN-13: 9780833043214
Publisher: RAND Corporation
Publication date: 04/08/2008
Pages: 276
Product dimensions: 6.08(w) x 9.13(h) x 0.58(d)

About the Author

Steven W. Popper (Ph.D., Economics, UC Berkeley) is a senior economist in RAND's International Policy Department and serves as Associate Director of the Science and Technology Policy Institute, as well as Professor of Science and Technology Policy in the RAND Graduate School.

Charles A. Goldman (PhD, Stanford University, Graduate School of Business-Economic Analysis and Policy) is an Economist at RAND and a professor of economics at the RAND Graduate School in Santa Monica, California.

Table of Contents

Preface iii

Figures xi

Tables xiii

Summary xv

Acknowledgments xxv

Abbreviations xxvii

Chapter 1 Introduction 1

Background 1

BK21 Program Structure 1

Phase I and Phase II Comparison 3

Significance of the Program 4

Objective of the Study 5

Approach 6

Outline of the Monograph 7

Chapter 2 Implications of Phase I Program Evaluations for Phase II Evaluations 9

Previous Performance Evaluations of BK21 Phase I 9

Annual and Interim Evaluations 9

Program-Level Evaluations 11

Findings from Previous Evaluations 14

Changes Resulting from BK21 Phase I 14

Suggestions for Improving the Evaluation System 16

Limitations of Previous Evaluations 17

Implications for Designing the Phase II Evaluation Model 19

Chapter 3 A Logic Model for Evaluating BK21 21

Literature Review and Focus Group Interviews 21

Developing the Logic Model 23

Identifying the Goals and Missions 26

Goals 27

Missions 29

Performance Milestones 31

Goals and Missions Identified for the Logic Model 33

Program Inputs 34

BK21 Funding 34

Strategy 37

Unit of Support 40

Recipient Selection Criteria 41

Performance Evaluation Criteria 42

Other Operational Mechanisms 42

Summary 43

Incentives, Activities, Outputs, and Outcomes of the Program 44

Incentives 44

Activities 47

Outputs 48

Outcomes 54

Other Observations 55

Summary 56

Chapter 4 Quantitative Model to Assess BK21 Program Outcomes 59

Introduction 59

Summary of Phase I Statistics 61

Potential Approaches 64

Unadjusted Comparisons of BK21 Winners and Losers 67

Ordinary Least Squares, Matching, and Propensity Score 69

Fixed-Effects and Difference-in-Differences Models 71

Regression Discontinuity74

Other Strategies 76

Quantitative Model 77

Heterogeneous Funding 80

The Power to Identify Impacts 81

Data for Use in Validating the Model 81

ISI Data 81

Ministry of Education Data 83

Validation of the Model 83

Pre-Treatment Trends 84

Another Possible Comparison Group: Nonapplicants 88

Validation 90

Variations on the Specification 96

Quality 97

Social Science and Humanities 100

Other Aspects of the Logic Model 100

Simultaneous Equations Methods 102

International Competitiveness 104

Simulation: Ability to Precisely Estimate Effects at Phase II Funding Levels 105

Simulation: Outside Funding 107

Implications for Data Collection 109

Summary 110

Chapter 5 Metrics and Measurement for Program Evaluation 111

General Issues of Measurement 112

Reliability 113

Validity 114

Direct and Indirect Measures 114

Cost Versus Benefit 115

Strategy for Framing Evaluation Measures 116

Matching Measures to Goals 117

Selecting Among Candidate Measures 118

Adding to Korea's Research Manpower Pool 132

Highly Educated Workforce Employment 134

High-Quality Globally Competitive Research 135

Measuring Quality of Research Output 135

Consistent Measures of Quality Within Fields 138

Measures of Quality Across All Fields 144

Relative Research Quality Measures 146

Changes to University/Department Infrastructure 146

Fostering a Competitive Environment Within Korea 147

Other Measures of Prestigious and Competitive Graduate Research 150

Measuring Enhanced University-Industry Collaboration 154

Measuring Enhanced Regional University Standing and Industry Linkage 156

Industrial Research Quantity, Quality, and National Labor Productivity 159

Chapter 6 Database Design to Support BK21 Program Evaluation and Measurement 161

Priority Among Proposed Measures 161

Database Design and Content 166

Format of Database Design Entries 173

Chapter 7 Future Development of BK21 and University Research in Korea 177

A Good Model for an Allocation System 178

Applying the Model to Korea 179

Research Project Funding 179

Portable Student Fellowships 181

Facilities and Infrastructure Funding 182

Promotion of Specific Regions or Fields 183

BK21 Program Management 183

Requirements for University and Department Governance 183

Changing the Korean System 184

Appropriate Scale and Number of Research Universities 185

Analogy to the U.S. System 185

Applying the Proposed Measurement Tools 189

Summary 191

Chapter 8 Conclusions 195

Directions for a Phase II Evaluation System 195

Phase II Program Goals and Characteristics Identified by the Logic Model 196

Quantitative Model and Data Requirements 196

Metrics, Measures, and Database to Assess Program Effects and to Monitor Progress 197

Assessing the Quality of Program Outputs and Outcomes 199

Quality of Research 199

Quality of the Highly Educated Workforce 200

Concentration Strategy and Competition Rule 201

Suggestions for Long-Term Changes in the Korean Scientific Research and Education System 202

Balancing Competition and Stability in Funding 202

Portable Student Fellowships 203

Infrastructure Funding 203

Improving University and Department Governance 204

Preconditions for the Proposed Changes 204

Enhancing Program Evaluation and Management Capabilities 205


A Performance Evaluation Metrics Used in Previous Evaluations of Phase I 207

B Interview Guides for Representatives of Participants, Policymakers, and Users Related to the University Research and Education System 209

C Selection Criteria for Applied Science Research Groups: Phase II 217

D Table with Lagged Outcomes 219

E Research Data Series 221

F Education Data Series 229

G Industry Data Series 237

H General Data Series 243

Bibliography 245

Customer Reviews

Most Helpful Customer Reviews

See All Customer Reviews