What You'll Build

Two helper functions that show up in nearly every layer of a neural network:

Linear takes an input vector and a weight matrix, multiplies each row of weights element-by-element with the input, and sums each row into a single output value:

input: [1, 2, 3] weights: [[0.1, 0.2, 0.3], row 0: 0.1*1 + 0.2*2 + 0.3*3 = 1.4 [0.4, 0.5, 0.6]] row 1: 0.4*1 + 0.5*2 + 0.6*3 = 3.2 output: [1.4, 3.2]

Enter fullscreen mode

Exit fullscreen mode

Two rows of weights means two output values. This is how neural networks change the size of data as it flows through layers.

Softmax takes a list of raw numbers and turns them into probabilities that add up to 1. For example, [2.0, 1.0, 0.1] becomes roughly [0.66, 0.24, 0.10] .…