Eigenvalues & Eigenvectors
Useful for PCA, spectral methods and understanding transformations.
1. Plain English Intuition
When a square matrix transforms space, most vectors get rotated and knocked off their line. An Eigenvector is a special vector that does NOT rotate—it stays pointing in the exact same direction, merely getting stretched or squashed. The Eigenvalue λ is the factor by which the eigenvector stretches: A v = λ v. In data analysis, the eigenvectors of a covariance matrix point in the directions of maximum data variation.
2. Where It Appears in ML & Deep Learning
- 1Principal Component Analysis (PCA): finding axes of maximum variance
- 2Spectral Graph Convolutional Networks (GCNs) and spectral clustering
- 3Analyzing dynamical systems and stability of recurrent neural networks (RNNs)
- 4Google PageRank algorithm (finding the principal eigenvector of the web transition matrix)
3. Inputs, Outputs, Shapes & Assumptions
4. What Can Go Wrong (Gotchas & Failures)
Attempting eigendecomposition on non-square matrices (use SVD instead).
Ignoring complex eigenvalues for non-symmetric matrices.
5. Python in Practice
Engineered using: np.linalg.eig, np.linalg.eigh (for symmetric matrices)
import numpy as np
# 2x2 symmetric covariance matrix
A = np.array([
[4.0, 1.0],
[1.0, 2.0]
])
# Compute eigenvalues and eigenvectors
eigenvalues, eigenvectors = np.linalg.eigh(A)
print(f"Eigenvalues: {eigenvalues}")
print(f"Principal eigenvector (max variance axis): {eigenvectors[:, 1]}")Fast Study Loop
Complete these 3 steps to mark this topic understood:
Curated Video Lesson
Eigenvectors and eigenvalues | Essence of linear algebra
Watch on YouTube