Menu

1 / 2

Chapter 11: The Full GPT - Assembling the Model

DEV Community·Gary Jackson·7 months ago
#bPArQTrJ
Reading 0:00
15s threshold

What You'll Build

Four files that together make the project complete:

Model.cs - the GptModel class that holds all parameters and implements the full forward pass (replacing the simplified Forward function from Chapters 6-7)

AdamOptimiser.cs - a reusable class wrapping the Adam state and update from Chapter 7

FullTraining.cs - the real training loop that uses GptModel across 10,000 steps

Program.cs - the finalised dispatcher with the full case wired up

Depends On

All previous chapters.

The GptModel Class

A few design notes before the code.

The Forward method takes a single token at a time, not the whole sequence at once. The KV cache (passed in as parameters) holds the context from previous positions. This is the same one-token-at-a-time approach from Chapter 9: we process tokens sequentially during both training and inference.

Each document or sample needs its own fresh KV cache.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.