Random Variables & Probability Distributions
Understand distributions as models of uncertainty; know discrete vs continuous and PMF/PDF/CDF at an intuitive level.
1. Plain English Intuition
Real-world data is noisy and imperfect. A random variable represents an outcome whose exact value is subject to chance (like tomorrow’s temperature or whether an email is spam). A probability distribution assigns probabilities to all possible values. Discrete variables use a Probability Mass Function (PMF, where probabilities sum to 1); continuous variables use a Probability Density Function (PDF, where area under the curve equals 1). In ML, your model does not output truth—it outputs parameters of a probability distribution.
2. Where It Appears in ML & Deep Learning
- 1Classification output heads (categorical distribution over class labels)
- 2Generative models (sampling tokens from a vocabulary distribution, diffusion models)
- 3Data generation and noise modeling (adding Gaussian noise in data augmentation)
- 4Uncertainty estimation in predictions (Bayesian neural networks, temperature scaling)
3. Inputs, Outputs, Shapes & Assumptions
4. What Can Go Wrong (Gotchas & Failures)
Confusing probability density with probability (PDF values can exceed 1.0; only the area must integrate to 1.0).
Uncalibrated probabilities: A neural network might output 0.99 confidence for an image that is out-of-distribution.
Sampling errors: Sampling from unnormalized logits without applying softmax or proper temperature scaling.
5. Python in Practice
Engineered using: scipy.stats, torch.distributions.Categorical, torch.distributions.Normal
from scipy import stats
import numpy as np
# 1. Continuous Distribution: Standard Normal Gaussian N(0, 1)
norm_dist = stats.norm(loc=0, scale=1)
pdf_at_zero = norm_dist.pdf(0.0) # Density at the mean
cdf_at_zero = norm_dist.cdf(0.0) # Probability of value <= 0 (0.50)
# 2. Discrete Distribution: Coin flip Bernoulli(p=0.7)
bern_dist = stats.bernoulli(p=0.7)
sample_draws = bern_dist.rvs(size=5)
print(f"Gaussian density at mean: {pdf_at_zero:.4f}")
print(f"Gaussian CDF at mean (P(X <= 0)): {cdf_at_zero:.2f}")
print(f"Bernoulli samples: {sample_draws}")Fast Study Loop
Complete these 3 steps to mark this topic understood: