Matrices and linear algebra

A matrix is a linear map written as a grid. $A\vec{x}$ transforms $\vec{x}$; everything in this page — determinants, row reduction, eigenvectors — is about understanding that map.

Matrix operations

For $A \in \mathbb{R}^{m \times n}$, $B \in \mathbb{R}^{n \times p}$: the product $AB \in \mathbb{R}^{m \times p}$ has entries

$(AB)_{ij} = \sum_k A_{ik} B_{kj} \quad \text{(row } i \text{ of } A \cdot \text{column } j \text{ of } B\text{)}$

• Not commutative: $AB \neq BA$ in general.
• Transpose: $(AB)^T = B^T A^T$.
• Inverse (2×2): $\begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} = \dfrac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$, exists iff $ad - bc \neq 0$.

Worked example.

$\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 3 \\ 4 & 7 \end{pmatrix}, \qquad \begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix}\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} = \begin{pmatrix} 3 & 4 \\ 4 & 6 \end{pmatrix}$

Same matrices, different order, different result.

Exercise 4.1

Let $A = \begin{pmatrix} 2 & -1 \\ 0 & 3 \end{pmatrix}$, $B = \begin{pmatrix} 1 & 1 \\ -2 & 0 \end{pmatrix}$.

a) Compute $AB$ and $BA$.

b) Compute $A^{-1}$ and verify $A A^{-1} = I$.

c) Solve $A\vec{x} = \begin{pmatrix} 1 \\ 6 \end{pmatrix}$ using $A^{-1}$.

Determinants

$\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$

3×3 by cofactor expansion along the first row:

$\det\begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix} = a(ei - fh) - b(di - fg) + c(dh - eg)$

Meaning: $|\det A|$ is the volume-scaling factor of the map $A$; $\det A = 0 \iff$ the map squashes space flat $\iff A$ has no inverse. Row swaps flip the sign; $\det(AB) = \det A \det B$.

Worked example.

$\det\begin{pmatrix} 2 & 0 & 1 \\ 1 & 3 & -1 \\ 0 & 2 & 1 \end{pmatrix} = 2(3 + 2) - 0 + 1(2 - 0) = 12$

Exercise 4.2

a) $\det\begin{pmatrix} 3 & -2 \\ 4 & 5 \end{pmatrix}$

b) $\det\begin{pmatrix} 1 & 2 & 0 \\ -1 & 1 & 3 \\ 2 & 0 & 1 \end{pmatrix}$

c) For which $\lambda$ is $\begin{pmatrix} 2-\lambda & 1 \\ 4 & 2-\lambda \end{pmatrix}$ singular? (This is a preview of eigenvalues.)

Systems of equations: row reduction

Write $A\vec{x} = \vec{b}$ as an augmented matrix and reduce with three legal moves: swap rows, scale a row, add a multiple of one row to another.

Worked example.

$\begin{cases} x + 2y - z = 3 \\ 2x + y + z = 3 \\ x - y + 2z = 0 \end{cases}$

$\left(\begin{array}{ccc|c} 1 & 2 & -1 & 3 \\ 2 & 1 & 1 & 3 \\ 1 & -1 & 2 & 0 \end{array}\right) \xrightarrow{R_2 - 2R_1,\; R_3 - R_1} \left(\begin{array}{ccc|c} 1 & 2 & -1 & 3 \\ 0 & -3 & 3 & -3 \\ 0 & -3 & 3 & -3 \end{array}\right)$

$R_3 = R_2$: one equation is redundant — infinitely many solutions. From $R_2$: $y = z + 1$; from $R_1$: $x = 1 - z$. Solution line: $(x, y, z) = (1, 1, 0) + t(-1, 1, 1)$.

Rank tells the story: rank 3 → unique solution; rank 2 with consistent system → a line of solutions; inconsistent row $(0\,0\,0\,|\,c)$, $c \neq 0$ → none.

Exercise 4.3

Solve by row reduction, by hand, showing the augmented matrix at each step:

a) $\begin{cases} 2x + y = 5 \\ x - 3y = -1 \end{cases}$

b) $\begin{cases} x + y + z = 6 \\ 2x - y + z = 3 \\ x + 2y - z = 2 \end{cases}$

c) $\begin{cases} x - y + 2z = 1 \\ 2x - 2y + 4z = 3 \end{cases}$ — how many solutions, and why?

Eigenvalues and eigenvectors

$\vec{v} \neq 0$ is an eigenvector of $A$ with eigenvalue $\lambda$ if

$A\vec{v} = \lambda\vec{v}$

— the map only stretches $\vec{v}$, never rotates it. Find $\lambda$ from the characteristic equation $\det(A - \lambda I) = 0$, then each $\vec{v}$ from $(A - \lambda I)\vec{v} = 0$.

Worked example. $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$

$\det(A - \lambda I) = (4-\lambda)(3-\lambda) - 2 = \lambda^2 - 7\lambda + 10 = (\lambda - 5)(\lambda - 2)$

$\lambda_1 = 5$: $(A - 5I)\vec{v} = 0 \Rightarrow \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix}\vec{v} = 0 \Rightarrow \vec{v}_1 = (1, 1)$.

$\lambda_2 = 2$: $\begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix}\vec{v} = 0 \Rightarrow \vec{v}_2 = (1, -2)$.

See it

The map $A$ applied to a ring of unit vectors. Eigenvectors (dashed) keep their direction — everything else turns.

import numpy as np
import matplotlib.pyplot as plt

A = np.array([[4.0, 1.0],
              [2.0, 3.0]])   # edit me

theta = np.linspace(0, 2*np.pi, 60)
circle = np.stack([np.cos(theta), np.sin(theta)])
image = A @ circle

vals, vecs = np.linalg.eig(A)
plt.plot(circle[0], circle[1], 'C0.', ms=3, label='unit circle')
plt.plot(image[0], image[1], 'C1.', ms=3, label='A · circle')
for i in range(2):
    v = vecs[:, i] * vals[i]
    plt.plot([0, v[0]], [0, v[1]], 'k--')
    plt.annotate(f'λ={vals[i]:.1f}', v)
plt.gca().set_aspect('equal'); plt.grid(True); plt.legend()
plt.show()

Exercise 4.4

a) Find the eigenvalues and eigenvectors of $B = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$.

b) Find the eigenvalues of $C = \begin{pmatrix} 2 & 1 \\ 0 & 2 \end{pmatrix}$ and all eigenvectors. What is geometrically special here?

c) Without computing: the matrix $R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ rotates by 90°. Argue why it has no real eigenvectors, then confirm via the characteristic polynomial.