Every part of an LLM was invented before 2018, each one to fix something that broke. Learn them in that order and they stop feeling arbitrary.
Four generations of AI, one sarcastic review. All four were sure it was positive.
Read this movie review:
"Wow, what a masterpiece. Two hours of pure brilliance, and I only checked my watch twenty times. The acting was so convincing that I truly believed the actors wanted to be somewhere else. A perfect film if you love naps."
You know in two seconds that it's a one-star review. In 2018, BERT, Google's best reading model at the time, fine-tuned on 1,000 movie reviews, was 98 % sure it was positive . A model that just counts words: 76 %. An averaging network: 99 %. An LSTM that reads word by word: 93 %. Four generations of reading machines, and not one of them gets the joke.
That review comes from David Louapre's lecture NLP before LLMs on Hugging Face, which builds every one of those models from scratch.…