1.3 Matrices
: 25 minutes
A matrix is a rectangular grid of numbers, much like a spreadsheet. In data work the convention is nearly universal: rows are records (one observation each) and columns are features (one measured attribute each).
Dragging our hypothetical BMI study a little further, imagine measuring the feature vector (weight, height, age) once for each of the 25 students in the class. We could keep 25 separate variables student1, student2, …, each a vector—but then every question about the data (“what is the average height?”) turns into a loop over 25 variable names. Stacking the 25 feature vectors as rows of one rectangular object is both cheaper to store and far easier to manipulate. That object is a matrix, and it looks like a table:
| student | weight | height | age |
|---|---|---|---|
| student 1 | 153 | 68 | 46 |
| student 2 | 196 | 55 | 30 |
| … | … | … | … |
| student 25 | 163 | 58 | 26 |
We denote matrices by bold capital letters (e.g., \bold{X}, \bold{Y}, and \bold{Z}). The expression \bold{A} \in \mathbb{R}^{m \times n} indicates that a matrix \bold{A} contains m \times n real-valued scalars, arranged as m rows and n columns. Alternatively, we say the size (or shape) of \bold{A} is (m,n).
When m = n, we say that a matrix is square. To refer to an individual element, we subscript both the row and column indices, e.g., a_{ij} is the value that belongs to \bold{A}’s i^{\textrm{th}} row and j^{\textrm{th}} column:
\bold{A}=\begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{bmatrix}. \tag{3.1}
Row first, then column. In a_{ij} the row index always comes first, and in code the outer index always selects a row. Memorise this now; almost every shape error you will hit this semester traces back to it.
A matrix can be read in two complementary ways, and switching between them freely is the skill this module is really teaching:
- as a stack of row vectors—one record per row, which is how a dataset looks; or
- as a list of column vectors—one feature per column, which is how 1.4 Matrix Operations will explain matrix multiplication and how the regression session builds its model.
Some Named Matrices
A handful of shapes come up so often that they have names. Let \bold{A} be square, of size n\times n.
- The zero matrix \bold{0} has every entry equal to 0 (it need not be square).
- \bold{A} is diagonal if a_{ij}=0 whenever i\neq j; only the entries a_{11},\ldots,a_{nn} may be non-zero.
- The identity matrix \bold{I}_n is the diagonal matrix whose diagonal entries are all 1. It plays the role of the number 1: \bold{I}\bold{A}=\bold{A}\bold{I}=\bold{A}.
- \bold{A} is upper triangular if a_{ij}=0 whenever i>j, and lower triangular if a_{ij}=0 whenever i<j.
- \bold{A} is symmetric if a_{ij}=a_{ji} for all i,j, i.e. it is unchanged by reflection across the diagonal.
\bold{I}_3=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix},\quad \bold{D}=\begin{bmatrix}2&0&0\\0&-1&0\\0&0&5\end{bmatrix},\quad \bold{U}=\begin{bmatrix}1&7&3\\0&4&2\\0&0&6\end{bmatrix},\quad \bold{S}=\begin{bmatrix}1&7&3\\7&4&2\\3&2&6\end{bmatrix}.
Symmetric matrices are not a curiosity: a correlation matrix and a covariance matrix are both symmetric, and so is the \bold{A}^T\bold{A} that shows up in the normal equations of regression.
Matrices in Python
Without NumPy, the closest thing Python offers is a list of lists: the outer list holds the rows, and each inner list is one row.
Indexing chains the two positions, outer first:
Getting a column is harder, and that asymmetry is the point. There is no X[:][1] shortcut that works—you have to visit every row:
Nothing in Python stops you from building a ragged list of lists whose rows have different lengths, which is not a matrix at all. Nothing checks it for you either—until NumPy next week, where rectangularity, shape, and column access all come for free.