Matrix Algebra for Linear Models


A self-contained introduction to matrix analysis theory and applications in the field of statistics

Comprehensive in scope, Matrix Algebra for Linear Models offers a succinct summary of matrix theory and its related applications to statistics, especially linear models. The book provides a unified presentation of the mathematical properties and statistical applications of matrices in order to define and manipulate data.

Written for theoretical ...

See more details below
Hardcover (New Edition)
$92.76 price
(Save 15%)$110.00 List Price
Other sellers (Hardcover)
  • All (7) from $88.04   
  • New (6) from $88.04   
  • Used (1) from $92.75   


A self-contained introduction to matrix analysis theory and applications in the field of statistics

Comprehensive in scope, Matrix Algebra for Linear Models offers a succinct summary of matrix theory and its related applications to statistics, especially linear models. The book provides a unified presentation of the mathematical properties and statistical applications of matrices in order to define and manipulate data.

Written for theoretical and applied statisticians, the book utilizes multiple numerical examples to illustrate key ideas, methods, and techniques crucial to understanding matrix algebra’s application in linear models. Matrix Algebra for Linear Models expertly balances concepts and methods allowing for a side-by-side presentation of matrix theory and its linear model applications. Including concise summaries on each topic, the book also features:

  • Methods of deriving results from the properties of eigenvalues and the singular value decomposition
  • Solutions to matrix optimization problems for obtaining more efficient biased estimators for parameters in linear regression models
  • A section on the generalized singular value decomposition
  • Multiple chapter exercises with selected answers to enhance understanding of the presented material

Matrix Algebra for Linear Models is an ideal textbook for advanced undergraduate and graduate-level courses on statistics, matrices, and linear algebra. The book is also an excellent reference for statisticians, engineers, economists, and readers interested in the linear statistical model.

Read More Show Less

Editorial Reviews

From the Publisher
“This is an excellent and comprehensive presentation of the use of matrices for linear models. The writing is very clear, and the layout is excellent. It would serve well either as a class text or as the foundation for individual personal study.” (International Statistical Review, 18 March 2014)
Read More Show Less

Product Details

  • ISBN-13: 9781118592557
  • Publisher: Wiley
  • Publication date: 1/13/2014
  • Edition description: New Edition
  • Edition number: 1
  • Pages: 392
  • Product dimensions: 6.20 (w) x 9.30 (h) x 1.00 (d)

Meet the Author

Marvin H. J. Gruber, PHD, is Professor Emeritus in the School of Mathematical Sciences at Rochester Institute of Technology. He has authored several books and journal articles in his areas of research interest, which include improving the efficiency of regression estimators. Dr. Gruber is a member of the American Mathematical Society and the American Statistical Association.

Read More Show Less

Table of Contents

Preface xiii

Acknowledgments xv

Part I Basic Ideas about Matrices and Systems of Linear Equations 1

Section 1 What Matrices are and Some Basic Operations with Them 3

1.1 Introduction, 3

1.2 What are Matrices and Why are they Interesting to a Statistician? 3

1.3 Matrix Notation, Addition, and Multiplication, 6

1.4 Summary, 10

Exercises, 10

Section 2 Determinants and Solving a System of Equations 14

2.1 Introduction, 14

2.2 Definition of and Formulae for Expanding Determinants, 14

2.3 Some Computational Tricks for the Evaluation of Determinants, 16

2.4 Solution to Linear Equations Using Determinants, 18

2.5 Gauss Elimination, 22

2.6 Summary, 27

Exercises, 27

Section 3 The Inverse of a Matrix 30

3.1 Introduction, 30

3.2 The Adjoint Method of Finding the Inverse of a Matrix, 30

3.3 Using Elementary Row Operations, 31

3.4 Using the Matrix Inverse to Solve a System of Equations, 33

3.5 Partitioned Matrices and Their Inverses, 34

3.6 Finding the Least Square Estimator, 38

3.7 Summary, 44

Exercises, 44

Section 4 Special Matrices and Facts about Matrices that will be Used in the Sequel 47

4.1 Introduction, 47

4.2 Matrices of the Form aIn + bJn, 47

4.3 Orthogonal Matrices, 49

4.4 Direct Product of Matrices, 52

4.5 An Important Property of Determinants, 53

4.6 The Trace of a Matrix, 56

4.7 Matrix Differentiation, 57

4.8 The Least Square Estimator Again, 62

4.9 Summary, 62

Exercises, 63

Section 5 Vector Spaces 66

5.1 Introduction, 66

5.2 What is a Vector Space?, 66

5.3 The Dimension of a Vector Space, 68

5.4 Inner Product Spaces, 70

5.5 Linear Transformations, 73

5.6 Summary, 76

Exercises, 76

Section 6 The Rank of a Matrix and Solutions to Systems of Equations 79

6.1 Introduction, 79

6.2 The Rank of a Matrix, 79

6.3 Solving Systems of Equations with Coefficient Matrix of Less than Full Rank, 84

6.4 Summary, 87

Exercises, 87

Part II Eigenvalues, the Singular Value Decomposition, and Principal Components 91

Section 7 Finding the Eigenvalues of a Matrix 93

7.1 Introduction, 93

7.2 Eigenvalues and Eigenvectors of a Matrix, 93

7.3 Nonnegative Definite Matrices, 101

7.4 Summary, 104

Exercises, 105

Section 8 The Eigenvalues and Eigenvectors of Special Matrices 108

8.1 Introduction, 108

8.2 Orthogonal, Nonsingular, and Idempotent Matrices, 109

8.3 The Cayley–Hamilton Theorem, 112

8.4 The Relationship between the Trace, the Determinant, and the Eigenvalues of a Matrix, 114

8.5 The Eigenvalues and Eigenvectors of the Kronecker Product of Two Matrices, 116

8.6 The Eigenvalues and the Eigenvectors of a Matrix of the Form aI + bJ, 117

8.7 The Loewner Ordering, 119

8.8 Summary, 121

Exercises, 122

Section 9 The Singular Value Decomposition (SVD) 124

9.1 Introduction, 124

9.2 The Existence of the SVD, 125

9.3 Uses and Examples of the SVD, 127

9.4 Summary, 134

Exercises, 134

Section 10 Applications of the Singular Value Decomposition 137

10.1 Introduction, 137

10.2 Reparameterization of a Non-full-Rank Model to a Full-Rank Model, 137

10.3 Principal Components, 141

10.4 The Multicollinearity Problem, 143

10.5 Summary, 144

Exercises, 145

Section 11 Relative Eigenvalues and Generalizations of the Singular Value Decomposition 146

11.1 Introduction, 146

11.2 Relative Eigenvalues and Eigenvectors, 146

11.3 Generalizations of the Singular Value Decomposition:

Overview, 151

11.4 The First Generalization, 152

11.5 The Second Generalization, 157

11.6 Summary, 160

Exercises, 160

Part III Generalized Inverses 163

Section 12 Basic Ideas about Generalized Inverses 165

12.1 Introduction, 165

12.2 What is a Generalized Inverse and How is One Obtained?, 165

12.3 The Moore–Penrose Inverse, 170

12.4 Summary, 173

Exercises, 173

Section 13 Characterizations of Generalized Inverses Using the Singular Value Decomposition 175

13.1 Introduction, 175

13.2 Characterization of the Moore–Penrose Inverse, 175

13.3 Generalized Inverses in Terms of the Moore–Penrose Inverse, 177

13.4 Summary, 185

Exercises, 186

Section 14 Least Square and Minimum Norm Generalized Inverses 188

14.1 Introduction, 188

14.2 Minimum Norm Generalized Inverses, 189

14.3 Least Square Generalized Inverses, 193

14.4 An Extension of Theorem 7.3 to Positive-Semi-definite Matrices, 196

14.5 Summary, 197

Exercises, 197

Section 15 More Representations of Generalized Inverses 200

15.1 Introduction, 200

15.2 Another Characterization of the Moore–Penrose Inverse, 200

15.3 Still Another Representation of the Generalized Inverse, 204

15.4 The Generalized Inverse of a Partitioned

Matrix, 207

15.5 Summary, 211

Exercises, 211

Section 16 Least Square Estimators for Less than Full-Rank Models 213

16.1 Introduction, 213

16.2 Some Preliminaries, 213

16.3 Obtaining the LS Estimator, 214

16.4 Summary, 221

Exercises, 221

Part IV Quadratic Forms and the Analysis of Variance 223

Section 17 Quadratic Forms and their Probability Distributions 225

17.1 Introduction, 225

17.2 Examples of Quadratic Forms, 225

17.3 The Chi-Square Distribution, 228

17.4 When does the Quadratic Form of a Random Variable have a Chi-Square Distribution?, 230

17.5 When are Two Quadratic Forms with the Chi-Square

Distribution Independent?, 231

17.6 Summary, 234

Exercises, 235

Section 18 Analysis of Variance: Regression Models and the One- and Two-Way Classification 237

18.1 Introduction, 237

18.2 The Full-Rank General Linear Regression Model, 237

18.3 Analysis of Variance: One-Way Classification, 241

18.4 Analysis of Variance: Two-Way Classification, 244

18.5 Summary, 249

Exercises, 249

Section 19 More ANOVA_253

19.1 Introduction, 253

19.2 The Two-Way Classification with Interaction, 254

19.3 The Two-Way Classification with One Factor Nested, 258

19.4 Summary, 262

Exercises, 262

Section 20 The General Linear Hypothesis 264

20.1 Introduction, 264

20.2 The Full-Rank Case, 264

20.3 The Non-full-Rank Case, 267

20.4 Contrasts, 270

20.5 Summary, 273

Exercises, 273

Part V Matrix Optimization Problems 275

Section 21 Unconstrained Optimization Problems 277

21.1 Introduction, 277

21.2 Unconstrained Optimization Problems, 277

21.3 The Least Square Estimator Again, 281

21.4 Summary, 283

Exercises, 283

Section 22 Constrained Minimization Problems with Linear Constraints 287

22.1 Introduction, 287

22.2 An Overview of Lagrange Multipliers, 287

22.3 Minimizing a Second-Degree Form with Respect to a Linear Constraint, 293

22.4 The Constrained Least Square Estimator, 295

22.5 Canonical Correlation, 299

22.6 Summary, 302

Exercises, 302

Section 23 The Gauss–Markov Theorem 304

23.1 Introduction, 304

23.2 The Gauss–Markov Theorem and the Least Square Estimator, 304

23.3 The Modified Gauss–Markov Theorem and the Linear Bayes Estimator, 306

23.4 Summary, 311

Exercises, 311

Section 24 Ridge Regression-Type Estimators 314

24.1 Introduction, 314

24.2 Minimizing a Second-Degree Form with Respect to a Quadratic Constraint, 314

24.3 The Generalized Ridge Regression Estimators, 315

24.4 The Mean Square Error of the Generalized Ridge Estimator without Averaging over the Prior Distribution, 317

24.5 The Mean Square Error Averaging over the Prior Distribution, 321

24.6 Summary, 321

Exercises, 321

Answers to Selected Exercises 324

References 366

Index 368

Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star


4 Star


3 Star


2 Star


1 Star


Your Rating:

Your Name: Create a Pen Name or

Barnes & Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation


  • - By submitting a review, you grant to Barnes & and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Terms of Use.
  • - Barnes & reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)