Practice Problems

Exercise 4.1 Explain the difference between linear dependence and independence of vectors. How can you determine if a set of vectors is linearly dependent in a specific vector space?

Exercise 4.2 Imagine you have a square matrix with all 1s on the diagonal and -1s everywhere else above and below the diagonal. What is the determinant of this matrix? Can you solve it without explicitly calculating every term?

Exercise 4.3 You’re working with a large dataset represented as a matrix. How can you efficiently reduce the dimensionality of this data using linear algebra techniques? Discuss the advantages and disadvantages of different dimensionality reduction methods.

Exercise 4.4 Given a matrix A, how can you determine if it is invertible without calculating the inverse directly?

Exercise 4.5 Explain the concept of eigenvalues and eigenvectors. How are they related to solving systems of linear equations and diagonalizing matrices?

Exercise 4.6 Describe a real-world scenario where linear regression might not be the most suitable approach for modeling a relationship between variables. Why not?

Exercise 4.7 How can you use linear algebra to solve a system of least squares problems? Explain the approach and its advantages for finding optimal solutions.

Exercise 4.8 Let’s say you have two matrices A and B. If the product AB is equal to the zero matrix, does it necessarily imply that either A or B is the zero matrix?

Exercise 4.9 Explain the concept of orthogonal projections. How can you find the projection of a vector onto a subspace?

Exercise 4.10 Imagine you are working on a portfolio optimization problem. You have historical returns data for various assets represented as a matrix. Can you use Eigenvalue Decomposition to construct an efficient frontier and identify optimal portfolios with desirable risk-return characteristics? If yes, how?

Exercise 4.11 Prove that the transpose of the transpose of a matrix is the matrix itself: (\mathbf{A}^\top)^\top = \mathbf{A}.

Exercise 4.12 Given two matrices \mathbf{A} and \mathbf{B}, show that sum and transposition commute: \mathbf{A}^\top + \mathbf{B}^\top = (\mathbf{A} + \mathbf{B})^\top.

Exercise 4.13 Given any square matrix \mathbf{A}, is \mathbf{A} + \mathbf{A}^\top always symmetric? Can you prove the result by using only the results of the previous two exercises?

Exercise 4.14 We defined the tensor X of shape (2, 3, 4) in this section. What is the output of len(X)? Write your answer without implementing any code, then check your answer using code.

Exercise 4.15 For a tensor X of arbitrary shape, does len(X) always correspond to the length of a certain axis of X? What is that axis?

Exercise 4.16 Run A / A.sum(axis=1) and see what happens. Can you analyze the results?

Exercise 4.17 When traveling between two points in downtown Manhattan, what is the distance that you need to cover in terms of the coordinates, i.e., in terms of avenues and streets? Can you travel diagonally?

Exercise 4.18 Consider a tensor of shape (2, 3, 4). What are the shapes of the summation outputs along axes 0, 1, and 2?

Exercise 4.19 Feed a tensor with three or more axes to the linalg.norm function and observe its output. What does this function compute for tensors of arbitrary shape?

Exercise 4.20 Consider three large matrices, say \mathbf{A} \in \mathbb{R}^{2^{10} \times 2^{16}}, \mathbf{B} \in \mathbb{R}^{2^{16} \times 2^{5}} and \mathbf{C} \in \mathbb{R}^{2^{5} \times 2^{14}}, initialized with Gaussian random variables. You want to compute the product \mathbf{A} \mathbf{B} \mathbf{C}. Is there any difference in memory footprint and speed, depending on whether you compute (\mathbf{A} \mathbf{B}) \mathbf{C} or \mathbf{A} (\mathbf{B} \mathbf{C}). Why?

Exercise 4.21 Consider three large matrices, say \mathbf{A} \in \mathbb{R}^{2^{10} \times 2^{16}}, \mathbf{B} \in \mathbb{R}^{2^{16} \times 2^{5}} and \mathbf{C} \in \mathbb{R}^{2^{5} \times 2^{16}}. Is there any difference in speed depending on whether you compute \mathbf{A} \mathbf{B} or \mathbf{A} \mathbf{C}^\top? Why? What changes if you initialize \mathbf{C} = \mathbf{B}^\top without cloning memory? Why?

Exercise 4.22 Consider three matrices, say \mathbf{A}, \mathbf{B}, \mathbf{C} \in \mathbb{R}^{100 \times 200}. Construct a tensor with three axes by stacking [\mathbf{A}, \mathbf{B}, \mathbf{C}]. What is the dimensionality? Slice out the second coordinate of the third axis to recover \mathbf{B}. Check that your answer is correct.