Skip to content
Optimal Lab

AI · interactive

A neural network, by hand

Each neuron multiplies its inputs by weights, adds a bias, and passes the result through an activation (here ReLU: negatives become 0). XOR, “one or the other but not both”, is impossible for one neuron but easy for two layers. Weights are hand-picked so you can check the maths.

11111-21x10x21h1 ≈ OR · bias 00h2 ≈ AND · bias -11output

h1 = relu(1 + 0 + 0) = 1 · h2 = relu(1 + 0 − 1) = 0 · y = 1 − 2×0 = 1

x1 x2XOR2-layer netbest single neuron
0 0000
0 1111
1 0111
1 1001

Hidden neuron 1 fires if either input is on (OR); neuron 2 only if both are (AND). The output is OR minus twice AND: on for exactly one input. A single neuron draws one straight line and can't separate XOR's cases.

Inputs (x1, x2)

  1. 1

  2. 2

  3. 3

    The hidden layer computes useful intermediate features (OR, AND); the next layer combines them.

    LLMs do this with billions of weights.

  4. 4

In production

Tools and technologies

Neural network libraries do this arithmetic at enormous scale.

Not tied to one language — use these names when you search docs or design a system.