Design and Modeling for Computer Experiments

by Kai-Tai Fang, Runze Li, Agus Sudjianto

Overview

Computer simulations based on mathematical models have become ubiquitous across the engineering disciplines and throughout the physical sciences. Successful use of a simulation model, however, requires careful interrogation of the model through systematic computer experiments. While specific theoretical/mathematical examinations of computer experiment design are available, those interested in applying proposed methodologies need a practical presentation and straightforward guidance on analyzing and interpreting experiment results.

Written by authors with strong academic reputations and real-world practical experience, Design and Modeling for Computer Experiments is exactly the kind of treatment you need. The authors blend a sound, modern statistical approach with extensive engineering applications and clearly delineate the steps required to successfully model a problem and provide an analysis that will help find the solution. Part I introduces the design and modeling of computer experiments and the basic concepts used throughout the book. Part II focuses on the design of computer experiments. The authors present the most popular space-filling designs - like Latin hypercube sampling and its modifications and uniform design - including their definitions, properties, construction and related generating algorithms. Part III discusses the modeling of data from computer experiments. Here the authors present various modeling techniques and discuss model interpretation, including sensitivity analysis. An appendix reviews the statistics and mathematics concepts needed, and numerous examples clarify the techniques and their implementation.

The complexity of real physical systems means that there is usually no simple analytic formula that sufficiently describes the phenomena. Useful both as a textbook and professional reference, this book presents the techniques you need to design and model computer experiments for practical problem solving.

Product Details

ISBN-13:	9780367578008
Publisher:	CRC Press
Publication date:	06/30/2020
Series:	Chapman & Hall/CRC Computer Science & Data Analysis
Pages:	302
Product dimensions:	6.12(w) x 9.19(h) x (d)

About the Author

Kai-Tai Fang, Runze Li, Agus Sudjianto

Table of Contents

Part I An Overview 1

1 Introduction 3

1.1 Experiments and Their Statistical Designs 3

1.2 Some Concepts in Experimental Design 4

1.3 Computer Experiments 10

1.3.1 Motivations 10

1.3.2 Metamodels 12

1.3.3 Computer Experiments in Engineering 16

1.4 Examples of Computer Experiments 20

1.5 Space-Filling Designs 24

1.6 Modeling Techniques 26

1.7 Sensitivity Analysis 31

1.8 Strategies for Computer Experiments and an Illustration Case Study 33

1.9 Remarks on Computer Experiments 38

1.10 Guidance for Reading This Book 40

Part II Designs for Computer Experiments 45

2 Latin Hypercube Sampling and Its Modifications 47

2.1 Latin Hypercube Sampling 47

2.2 Randomized Orthogonal Array 51

2.3 Symmetric and Orthogonal Column Latin Hypercubes 54

2.4 Optimal Latin Hypercube Designs 60

2.4.1 IMSE Criterion 60

2.4.2 Entropy Criterion 62

2.4.3 Minimax and Maximin Distance Criteria and Their extension 64

2.4.4 Uniformity Criterion 65

3 Uniform Experimental Design 67

3.1 Introduction 67

3.2 Measures of Uniformity 68

3.2.1 The Star L_p-Discrepancy 68

3.2.2 Modified L₂Discrepancy 70

3.2.3 The Centered Discrepancy 71

3.2.4 The Wrap-Around Discrepancy 72

3.2.5 A Unified Definition of Discrepancy 73

3.2.6 Descrepancy for Categorical Factors 75

3.2.7 Applications of Uniformity in Experimental Designs 76

3.3 Construction of Uniform Designs 78

3.3.1 One-Factor Uniform Designs 78

3.3.2 Symmetrical Uniform Designs 79

3.3.3 Good Lattice Point Method 80

3.3.4 Latin Square Method 85

3.3.5 Expanding Orthogonal Array Method 86

3.3.6 The Cutting Method 86

3.3.7 Construction of Uniform Designs by Optimization 90

3.4 Characteristics of the Uniform Design: Admissibility, Minimaxity, and Robustness 90

3.5 Construction of Uniform Designs via Resolvable Balanced Incomplete Block Designs 93

3.5.1 Resolvable Balanced Incomplete Block Designs 93

3.5.2 RBIBD Construction Method 94

3.5.3 New Uniform Designs 94

3.6 Construction of Asymmetrical Uniform Designs 97

3.6.1 Pseudo-Level Technique 97

3.6.2 Collapsing Method 97

3.6.3 Combinatorial Method 100

3.6.4 Miscellanea 103

4 Optimization in Construction of Designs for Computer Experiments 105

4.1 Optimization Problem in Construction of Designs 105

4.1.1 Algorithmic Construction 106

4.1.2 Neighborhood 106

4.1.3 Replacement Rule 107

4.1.4 Iteration Formulae 109

4.2 Optimization Algorithms 113

4.2.1 Algorithms 113

4.2.2 Local Search Algorithm 114

4.2.3 Simulated Annealing Algorithm 115

4.2.4 Threshold Accepting Algorithm 115

4.2.5 Stochastic Evolutionary Algorithm 116

4.3 Lower Bounds of the Discrepancy and Related Algorithm 117

4.3.1 Lower Bounds of the Categorical Discrepancy 119

4.3.2 Lower Bounds of the Wrap-Around L₂-Discrepancy 119

4.3.3 Lower Bounds of the Centered L₂-Discrepancy 121

4.3.4 Balance-Pursuit Heuristic Algorithm 122

Part III Modeling for Computer Experiments 125

5 Metamodeling 127

5.1 Basic Concepts 127

5.1.1 Mean Square Error and Prediction Error 127

5.1.2 Regularization 130

5.2 Polynomial Models 133

5.3 Spline Method 139

5.3.1 Construction of Spline Basis 140

5.3.2 An Illustration 142

5.3.3 Other Bases of Global Approximation 144

5.4 Gaussian Kriging Models 145

5.4.1 Prediction via Kriging 146

5.4.2 Estimation of Parameters 147

5.4.3 A Case Study 153

5.5 Bayesian Approach 159

5.5.1 Gaussian Processes 159

5.5.2 Bayesian Prediction of Deterministic Functions 160

5.5.3 Use of Derivatives in Surface Prediction 162

5.5.4 An Example: Borehole Model 165

5.6 Neural Network 167

5.6.1 Multi-Layer Perceptron Networks 168

5.6.2 A Case Study 172

5.6.3 Radial Basis Functions 177

5.7 Local Polynomial Regression 180

5.7.1 Motivation of Local Polynomial Regression 180

5.7.2 Metamodeling via Local Polynomial Regression 183

5.8 Some Recommendations 184

5.8.1 Connections 184

5.8.2 Recommendations 185

6 Model Interpretation 187

6.1 Introduction 187

6.2 Sensitivity Analysis Based on Regression Analysis 188

6.2.1 Criteria 188

6.2.2 An Example 191

6.3 Sensitivity Analysis Based on Variation Decomposition 193

6.3.1 Functional ANOVA Representation 193

6.3.2 Computational Issues 195

6.3.3 Example of Sobol' Global Sensitivity 198

6.3.4 Correlation Ratios and Extension of Sobol' Indices 199

6.3.5 Fourier Amplitude Sensitivity Test 202

6.3.6 Example of FAST Application 205

7 Functional Response 207

7.1 Computer Experiments with Functional Response 207

7.2 Spatial Temporal Models 215

7.2.1 Functional Response with Sparse Sampling Rate 215

7.2.2 Functional Response with Intensive Sampling Rate 218

7.3 Penalized Regression Splines 219

7.4 Functional Linear Models 222

7.4.1 A Graphical Tool 223

7.4.2 Efficient Estimation Procedure 224

7.4.3 An Illustration 226

7.5 Semiparametric Regression Models 230

7.5.1 Partially Linear Model 230

7.5.2 Partially Functional Linear Models 234

7.5.3 An Illustration 236

Appendix 241

A.1 Some Basic Concepts in Matrix Algebra 241

A.2 Some Concepts in Probability and Statistics 244

A.2.1 Random Variables and Random Vectors 244

A.2.2 Some Statistical Distributions and Gaussian Process 247

A.3 Linear Regression Analysis 249

A.3.1 Linear Models 250

A.3.2 Method of Least Squares 251

A.3.3 Analysis of Variance 252

A.3.4 An Illustration 253

A.4 Variable Selection for Linear Regression Models 256

A.4.1 Nonconvex Penalized Least Squares 257

A.4.2 Iteratively Ridge Regression Algorithm 258

A.4.3 An Illustration 259

Acronyms 261

References 263

Index 283

Author Index 287

Editorial Reviews

". . . very well-organized text . . . makes a very valuable contribution to the field. I highly recommend it for anyone trying to learn design and modeling techniques for computer experiments. In particular, it will be a useful professional reference for scientists and engineers in practicing computer experiments, a comprehensive resource book for statisticians interested in developing new techniques for designing and modeling computation experiments, and an excellent book for undergraduate and graduate students. The authors’ careful and thorough presentation style makes the book a very enjoyable read."

– Hao Helen Zhang, North Carolina State University, in JASA, December 2008

From the Publisher