They don't. They join many straight lines into shapes, and that's where a network's power comes from.
A single neuron can only draw one straight line. Yet neural networks recognize faces, read handwriting, and trace curved boundaries everywhere. So where do the curves come from?
The surprising answer: from straight lines, lots of them, combined by a hidden layer.
One line can't fence in a cluster
A neuron splits the plane in two with a straight line. Points on one side get a "yes," and points on the other side get a "no."
That works when the classes sit on opposite sides of the page. But look at this case: the blue points form a cluster in the middle, surrounded by the others. Wherever you draw one straight line, it can't fence the cluster in.
The blue points sit in the middle. A single straight line can't separate them from the points around them.
Add a hidden layer
So add a layer of neurons between the inputs and the output. This is a hidden layer.…