My new video, In 19 Minutes, You'll Understand 10 Years of LLMs , follows one question through every era of language models: "What is the capital of France?"
This post is the developer version. It goes through every chapter of the video with the actual terms and numbers. Where the video shows a model's output, we ran an open model ourselves, so you get the code to reproduce it.
pip install torch transformers accelerate
Enter fullscreen mode
Exit fullscreen mode
All outputs below are what we got on an RTX 4070 Ti. Yours may differ slightly with hardware and precision.
TL;DR timeline
Year Milestone What it fixed What it left open
1948–51 Shannon's next-letter game "predict what comes next" as a measurable idea no machine could play it well
2013 word2vec words become vectors that capture meaning one vector per word, whatever the context
1997–2014 RNN, LSTM, seq2seq read sentences, translate one fixed-size summary per sentence
2014 attention (Bahdanau) the decoder looks back at every word still reads…