In Part 1 , I focused on understanding how a neural network learns.

I built neurons, experimented with weights and biases, implemented gradient descent, and worked through backpropagation. I also learned how easily a model can go wrong when gradients or training parameters aren't handled correctly.

But there was still one question I hadn't answered:

How do we get a neural network to work with language?

A model that learns to add two numbers is one thing. A model that reads a sentence, considers the surrounding words and predicts what should come next is a completely different challenge.

I started exploring the architecture that made modern language models possible: the Transformer.

And before I could understand its attention mechanism, I had to understand how text becomes something a neural network can process.

1.…