Modern regression and related methods have been developed using notation and concepts of matrix algebra. Indeed, many of the underlying theoretical results could not have been developed without matrix algebra. Although we have, as much as possible, developed most of the concepts in this book without resorting to matrix notation, some of the more advanced topics require presenting the equations in matrix notation. In addition, most of the results associated with multiple regression and related topics can be presented more compactly and simply using matrix notation. This appendix reviews the basic definitions and notation necessary to understand the use of matrices in this book. We will concentrate on notational, as opposed to computational or theoretical, issues because computers take care of the computations and there are many excellent books on the theory of linear algebra for those who would like to understand linear algebra in more detail.

A *scalar* is an ordinary number, such as 17.

A *matrix* is a rectangular array of numbers with *r* rows and *c* columns. For example, let **X** be the 4 × 3 matrix

Each number in the matrix is called an *element.* Typically (but not always), the matrix is denoted with a boldface letter and the elements are denoted with the same letter in regular type. The subscripts on the symbol for the element denote its row and column. *x _{ij}* is the element in the

*i*th row and

*j*th column. For example, the element

*x*is the element in the second row and the third column, 9.

_{23}A *vector* is a matrix with one column. For example, let **y** be the 4 × 1 matrix, or 4 vector

Because all vectors have a single column, we only need one subscript to define an element in a vector. For example, the first element in the vector ** y** is

*y*= 17. There are no fundamental differences between matrices and vectors; a vector is just a matrix with one column.

_{1}A *square matrix* is one in which the number of rows equals the number of columns. A square matrix **S** is *symmetric* if *s _{ij}* =

*s*. A square matrix

_{ji}**D**is

*diagonal*if all the off-diagonal elements are zero, that is, if

*d*= 0 unless

_{ij}*i*=

*j*. For example,

is a 3 × 3 symmetric matrix, and

is a 3 × 3 diagonal matrix. A diagonal matrix in which all the diagonal elements are 1 is called the *identity matrix* and denoted with **I**. For example, the 4 × 4 identity matrix is

The identity matrix plays a role similar to 1 in scalar arithmetic.