PCA (Principal Component Analysis)
Useful for dimensionality reduction, visualization and decorrelation.
1. Plain English Intuition
PCA is an unsupervised technique that compresses high-dimensional data (e.g. 100 features) down to a few principal components (e.g. 2 or 3) while retaining as much information (variance) as possible. It rotates your coordinate axes to align with the directions of greatest spread in the data. The first principal component captures the most variance, the second captures the next most while being strictly orthogonal (uncorrelated) to the first, and so on.
2. Where It Appears in ML & Deep Learning
- 1High-dimensional data visualization (e.g. projecting 768-dim embeddings to 2D scatter plots)
- 2Preprocessing pipeline step to eliminate multicollinearity and reduce compute costs
- 3Noise filtering in image and sensor signals
- 4Gene expression analysis and facial recognition (Eigenfaces)
3. Inputs, Outputs, Shapes & Assumptions
4. What Can Go Wrong (Gotchas & Failures)
Applying PCA without standardizing features first (a single large-scale feature will dominate the first component).
Assuming PCA works well on non-linear manifolds (use UMAP or t-SNE for complex non-linear visualization).
5. Python in Practice
Engineered using: sklearn.decomposition.PCA
from sklearn.decomposition import PCA
import numpy as np
# Synthetic dataset: 50 samples with 5 features
X = np.random.randn(50, 5)
# Compress to 2 principal components
pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X)
print(f"Original shape: {X.shape} -> Reduced shape: {X_reduced.shape}")
print(f"Explained Variance Ratio: {pca.explained_variance_ratio_}")
print(f"Total variance retained: {np.sum(pca.explained_variance_ratio_):.2%}")Fast Study Loop
Complete these 3 steps to mark this topic understood:
Curated Video Lesson
Principal Component Analysis (PCA) Step-by-Step
Watch on YouTube