Kumita ← Open visual lesson

Why Layers Need a Bend

The code behind the picture

Follow the lesson’s implementation, one code cell at a time.

Choose a language and read the cells in order. These are small teaching examples; the visual lesson runs separately.

Follow the calculation

Same inputs · Same operation · Same result

3 teaching cells · Python standard library · Read-only

1. The four plates and the network
Each plate is (a, b): 1 if that switch is up. The weights are chosen by hand: two hidden units, a − b and b − a, then their sum.
In [ ]:
```
import json

plates = [(0, 0), (1, 0), (0, 1), (1, 1)]
w1 = [[1, -1], [-1, 1]]
w2 = [1, 1]
```

2. Forward pass with and without ReLU

Compute the hidden scores, optionally keep only positive ones, and add them.

def forward(x, bend):
    z = [row[0] * x[0] + row[1] * x[1] for row in w1]
    h = [max(0, v) for v in z] if bend else z
    return h[0] * w2[0] + h[1] * w2[1]

with_relu = [forward(p, True) for p in plates]
without_relu = [forward(p, False) for p in plates]

3. Inspect the result
With ReLU the outputs are 0, 1, 1, 0, the lamp rule. Without it the whole network is the constant zero.
In [ ]:
```
print(json.dumps([with_relu, without_relu]))
```

Expected result

Running the three cells in order produces this result in either language.