What You'll Build
Four files that together make the project complete:
Model.cs - the GptModel class that holds all parameters and implements the full forward pass (replacing the simplified Forward function from Chapters 6-7)
AdamOptimiser.cs - a reusable class wrapping the Adam state and update from Chapter 7
FullTraining.cs - the real training loop that uses GptModel across 10,000 steps
Program.cs - the finalised dispatcher with the full case wired up
Depends On
All previous chapters.
The GptModel Class
A few design notes before the code.
The Forward method takes a single token at a time, not the whole sequence at once. The KV cache (passed in as parameters) holds the context from previous positions. This is the same one-token-at-a-time approach from Chapter 9: we process tokens sequentially during both training and inference.
Each document or sample needs its own fresh KV cache.…