Logistic Regression, Sigmoid, Logits & Probabilities
Understand how a linear score becomes a binary probability and what logits mean.
1. Plain English Intuition
Linear regression outputs numbers from -∞ to +∞, which makes no sense for predicting probabilities. Logistic regression solves this by taking that raw linear score (called the Logit, z = w^T x + b) and passing it through the Sigmoid activation function: σ(z) = 1 / (1 + e^(-z)). This smoothly squashes any real value into the valid probability range (0, 1). If z > 0, probability > 0.5 (predict Class 1); if z < 0, probability < 0.5 (predict Class 0).
2. Where It Appears in ML & Deep Learning
- 1Standard baseline for binary classification (spam vs ham, fraud vs valid, click vs no-click)
- 2Binary output heads in neural networks (torch.sigmoid)
- 3CTR (Click-Through Rate) prediction models in ad tech
- 4Evaluating calibration and ROC-AUC metrics
3. Inputs, Outputs, Shapes & Assumptions
4. What Can Go Wrong (Gotchas & Failures)
Separable classes: If a feature perfectly separates classes, weights explode toward ±infinity trying to make probabilities 0.0 or 1.0 (requires L2 regularization).
Confusing logits (unbounded real numbers) with probabilities (values between 0 and 1).
Using an arbitrary 0.5 classification threshold when classes are imbalanced (e.g. 99% class 0).
5. Python in Practice
Engineered using: sklearn.linear_model.LogisticRegression, torch.sigmoid
import numpy as np
def sigmoid(z):
return 1.0 / (1.0 + np.exp(-z))
# Model parameters
w = np.array([1.8, -0.9])
b = -0.5
# Feature vector for a candidate transaction
x = np.array([2.0, 1.0])
# Compute raw logit score
logit = np.dot(w, x) + b
# Squash logit to probability
prob = sigmoid(logit)
print(f"Raw Logit: {logit:.2f}")
print(f"Predicted Probability P(y=1): {prob:.4f}")
print(f"Classification (>0.5): {int(prob > 0.5)}")Fast Study Loop
Complete these 3 steps to mark this topic understood:
Curated Video Lesson
Logistic Regression Details and Log-Odds
Watch on YouTube