Mathematics for Applied Machine Learning
The pragmatic syllabus for building, understanding, and debugging real models. Zero symbolic gymnastics. Focus on tensor geometry, gradient sensitivity, loss objectives, and numerical stability.
Understanding > Hand Calculation
Never waste time manually expanding determinants or hand-calculating derivatives. Understand plain English intuition, tensor shapes, assumptions, and libraries.
The One Python Warning
Libraries do arithmetic, but they cannot save you from the wrong loss, mismatched distributions, unscaled features, or floating-point underflow.
The 3-Step Fast Study Loop
Watch curated 10m video → Explain in 2 sentences → Run one tiny Python snippet → move on!
Curriculum Progress
Functions, Calculus & Optimization Gradients
Rates of change, backpropagation sensitivity flow, and parameter updates.
Functions, Exponentials & Logarithms
Understand why exp/log appear in sigmoid, softmax, likelihood, and cross-entropy. No log-manipulation drills.
np.exp, np.log, torch.exp, torch.logDerivatives, Partial Derivatives & Gradients
Derivative = sensitivity/rate of change; gradient = direction of steepest increase across many parameters.
loss.backward(), torch.autograd.grad, jax.gradChain Rule & Backpropagation Intuition
Understand how local derivatives compose so error signals can flow backward through a model.
loss.backward(), jax.vjp, autograd graphGradient Descent, SGD & Learning Rate
Understand parameter updates, step size, minibatches, convergence problems, and why training can diverge.
torch.optim.SGD, torch.optim.AdamW, lr_schedulerLinear Algebra, Tensors & Geometry
Multi-dimensional arrays, spatial transformations, dot products, and projections.
Vectors, Matrices, Tensors & Shapes
Know what a vector/matrix/tensor represents and how dimensions/shapes map to samples, features, batches, and parameters.
np.array, torch.Tensor, .shape, .view(), .reshape()Matrix Multiplication
Understand it as combining linear transformations / weighted features; be able to reason about compatible shapes.
A @ B, np.matmul, torch.matmul, torch.bmmDot Product, Projection, Norms, Distance & Cosine Similarity
Understand similarity, vector length and distance. You do not need repetitive hand calculations.
np.dot, np.linalg.norm, torch.cosine_similarity, sklearn.metrics.pairwise.cosine_similarityProbability, Distributions & Estimation
Quantifying uncertainty, expectation, covariance, Bayes theorem, and maximum likelihood.
Random Variables & Probability Distributions
Understand distributions as models of uncertainty; know discrete vs continuous and PMF/PDF/CDF at an intuitive level.
scipy.stats, torch.distributions.Categorical, torch.distributions.NormalExpected Value, Mean, Variance & Standard Deviation
Understand center, spread and long-run average; recognize why expected loss is central to ML.
np.mean, np.var, np.std, torch.mean, torch.var_meanCovariance & Correlation
Understand how variables move together, what sign/magnitude mean, and why correlation is not causation.
np.cov, np.corrcoef, pandas.DataFrame.corr()Conditional Probability, Independence & Bayes' Theorem
Understand updating beliefs from evidence, priors/posteriors, and independence assumptions.
sklearn.naive_bayes.GaussianNB, scipy.statsCommon Distributions: Bernoulli, Categorical & Gaussian
Know what kind of target/noise each models; recognize their shapes and parameters.
scipy.stats.bernoulli, scipy.stats.multinomial, scipy.stats.norm, torch.distributionsLikelihood, Log-Likelihood & Maximum Likelihood Estimation (MLE)
Know probability-vs-likelihood and why fitting often means maximizing log-likelihood.
Usually inside .fit() / torch.nn.NLLLoss, scipy.optimize.minimizeSampling, Central Limit Theorem & Standard Error
Understand why estimates fluctuate between samples and why averages often become approximately normal.
scipy.stats.sem, np.random.choice, bootstrap utilitiesMachine Learning Models & Loss Functions
Translating predictions into penalties: least squares, logistic curves, cross-entropy, and softmax.
Least Squares & Linear Regression
Understand residuals, squared error, and what the fitted coefficients mean.
sklearn.linear_model.LinearRegression, np.linalg.lstsqLogistic Regression, Sigmoid, Logits & Probabilities
Understand how a linear score becomes a binary probability and what logits mean.
sklearn.linear_model.LogisticRegression, torch.sigmoidLoss Functions: MSE, Binary Log-Loss & Cross-Entropy
Know what the model is optimizing and why regression/classification use different losses.
mean_squared_error, log_loss, torch.nn.MSELoss, torch.nn.CrossEntropyLoss, torch.nn.BCEWithLogitsLossSoftmax & Multiclass Probabilities
Understand turning multiple logits into a probability distribution over classes.
torch.softmax, scipy.special.softmax, torch.log_softmaxGeneralization, Regularization & Numerical Stability
Bias-variance trade-offs, L1/L2 penalties, data leakage prevention, and floating-point health.
Bias–Variance, Underfitting & Overfitting
Understand the generalization trade-off and why lower training error is not always better.
sklearn.model_selection.learning_curve, validation metricsRegularization: L1 & L2
Understand why parameter penalties can improve generalization and the qualitative difference between L1 and L2.
Ridge, Lasso, ElasticNet, optimizer weight_decayTrain/Validation/Test Splits & Cross-Validation
Understand unbiased evaluation, model selection, and data leakage.
train_test_split, KFold, StratifiedKFold, cross_val_scoreFeature Scaling, Standardization & Numerical Stability
Know why scale changes optimization/distance methods; understand overflow/underflow and why stable library implementations matter.
StandardScaler, MinMaxScaler, scipy.special.logsumexp, torch.logsumexpGood to Do — On-Demand Mathematical Extensions
Learn when a concrete model, dimensionality reduction, or probabilistic project requires it.
Linear Combinations, Span, Basis & Rank
Useful for dimensionality, redundant features and representation intuition.
np.linalg.matrix_rank, scipy.linalg.orthnp.linalg.eig, np.linalg.eigh (for symmetric matrices)PCA (Principal Component Analysis)
Useful for dimensionality reduction, visualization and decorrelation.
sklearn.decomposition.PCASingular Value Decomposition (SVD)
Useful for low-rank approximation, PCA, recommender systems and embeddings.
np.linalg.svd, sklearn.decomposition.TruncatedSVD, torch.svdMatrix Inverse & Determinant — Concept Only
Know what they mean; prefer solvers/decompositions in real code.
np.linalg.det, np.linalg.solve (preferred over inv), np.linalg.slogdetJacobians & Hessians — Concept Only
Useful when reading optimization/autodiff literature or debugging advanced models.
torch.autograd.functional.jacobian, torch.autograd.functional.hessianEntropy, Cross-Entropy & KL Divergence
Especially useful for deep learning, generative/probabilistic models and information theory.
scipy.stats.entropy, torch.nn.KLDivLossConvexity & Constrained Optimization
Useful for SVMs, classical optimization and understanding guarantees.
scipy.optimize.minimize, cvxpyConfidence Intervals & Hypothesis Testing
Important if your job includes experimentation, A/B testing or scientific inference.
scipy.stats.ttest_ind, statsmodels.stats.weightstatspymc, pyro, stanFourier Transform & Convolution Intuition
Useful for signals, vision, audio and some time-series work.
np.fft.fft, scipy.signal.spectrogram, torch.fftstatsmodels.tsa.stattools.acf, statsmodels.tsa.stattools.adfullerscipy.stats, gymnasium, stable-baselines3