Software 3.0, English as code, and the new grammar of programming with LLMs.

Originally published on Lei Hua's Substack .

Anchors:

2025-02-05 · Deep Dive into LLMs like ChatGPT · https://www.youtube.com/watch?v=7xTGNNLPyMI

2025-02-28 · How I use LLMs · https://www.youtube.com/watch?v=EWvNQjAaOHw

2025-06-17 · Software Is Changing (Again) · YC AI Startup School · https://www.youtube.com/watch?v=LCEmiRjPEtQ

Epigraph

"Software 1.0: humans write explicit code.

Software 2.0: humans create datasets, objectives, and neural networks; the program is learned into weights.

Software 3.0: humans program LLMs through prompts, context, tools, examples, memory, and instructions."

— Andrej Karpathy, Sequoia Ascent 2026 summary (recapitulating the YC 2025 talk)

I. Teaching LLMs the Second Time

On February 5, 2025, Karpathy posted a 3-hour-31-minute video to his own YouTube channel titled Deep Dive into LLMs like ChatGPT . It was the upgrade of his November 2023 Intro to Large Language Models .…