Your First Neuron

The Story

Everything in deep learning — from the smallest classifier to GPT-4 — is built from one structure: the neuron. Today you will build one with nothing but Python. No libraries. No frameworks. Just you and the math.

What a Neuron Actually Does

A neuron is a function. It takes inputs, scales them by weights, adds a bias, and squashes the result through an activation function.

output = activation(w1*x1 + w2*x2 + ... + wn*xn + bias)

That is it. Every fancy model you have ever heard of is just billions of these wired together.

Step 1: The Math (No Code Yet)

Imagine a neuron with two inputs:

• Input 1: x1 = 2.0, weight w1 = 0.5
• Input 2: x2 = 3.0, weight w2 = -1.0
• Bias: b = 1.0

The weighted sum is:

sum = (0.5 * 2.0) + (-1.0 * 3.0) + 1.0 = 1.0 - 3.0 + 1.0 = -1.0

Then we pass it through a simple activation function. Let's use the ReLU (Rectified Linear Unit):

ReLU(x) = max(0, x)

So ReLU(-1.0) = 0. The neuron "fires" nothing.

Step 2: Build It in Python

Open the code editor below and type this:

def relu(x):
    return max(0, x)

def neuron(inputs, weights, bias):
    total = sum(w * xi for w, xi in zip(weights, inputs))
    total += bias
    return relu(total)

# Try it
inputs = [2.0, 3.0]
weights = [0.5, -1.0]
bias = 1.0
print(neuron(inputs, weights, bias))

Key Insight

The weight tells the neuron how much to care about an input. The bias tells it how easy it is to fire. Together, they are the only tunable knobs in the entire universe of neural networks.

What Is Actually Happening Here?

You have just implemented the core computation of every neural network ever built. When ChatGPT generates text, it is doing this exact operation — just billions of times in parallel, with learned weights.

The magic is not in the operation. The magic is in finding the right weights. That is what we will tackle next.

Retrieval Check

Close your eyes (seriously). What are the three mathematical operations a neuron performs, in order?