Menu

You Will Understand 10 Years of LLMs
📰

You Will Understand 10 Years of LLMs

DEV Community: nlp·Walid Hadri·about 7 hours ago
#s60bfrgl
#devtonlp#devto#dev#model#token
Reading 0:00
15s threshold

My new video, In 19 Minutes, You'll Understand 10 Years of LLMs , follows one question through every era of language models: "What is the capital of France?"

This post is the developer version. It goes through every chapter of the video with the actual terms and numbers. Where the video shows a model's output, we ran an open model ourselves, so you get the code to reproduce it.

pip install torch transformers accelerate

Enter fullscreen mode

Exit fullscreen mode

All outputs below are what we got on an RTX 4070 Ti. Yours may differ slightly with hardware and precision.

TL;DR timeline

Year Milestone What it fixed What it left open

1948–51 Shannon's next-letter game "predict what comes next" as a measurable idea no machine could play it well

2013 word2vec words become vectors that capture meaning one vector per word, whatever the context

1997–2014 RNN, LSTM, seq2seq read sentences, translate one fixed-size summary per sentence

2014 attention (Bahdanau) the decoder looks back at every word still reads…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.