Vectors and Matrices
“It is not enough to have a good mind; the main thing is to use it well.”
– René Descartes
Linear algebra lies at the core of data science. Simply put, it is the study of vectors, vector spaces, and linear transformations—the last of which we will represent by matrices. Data is stored and manipulated as matrices, and more generally as tensors. The subject also supplies the techniques that extract structure from those tables: LU decomposition, principal component analysis, low-rank approximation. Modern machine learning is written in this language, and a model that seems mysterious in words is usually obvious once written as a matrix. This chapter is a gentle introduction to the two building blocks: vectors and matrices.
By the end of this chapter you should be able to:
Read and write the notation for scalars, vectors, and matrices, and state the shape of each.
Carry out vector and matrix operations by hand—addition, scaling, dot product, transpose, multiplication, inverse—and say what each one does geometrically.
Compute a dot product and recognise orthogonality, the ideas the whole module is built on.
Explain why a matrix is a linear map, and why A\bold{x} is a linear combination of the columns of A.
Represent all of this in plain Python with
list(oflists), and see clearly where plain Python runs out.
This is the first week of the Thinking in Arrays module, and two later sessions lean on it directly.
- The dot product and orthogonality of 1.2 become projection in 3.2 Projection and Orthogonality.
- “A matrix is a linear map, and A\bold{x} combines the columns of A” from 1.4 becomes the geometry of linear regression in the Nov 3 session, where the fitted values turn out to be a projection onto a column space.
Nothing here is throat-clearing. Spend the time now.