tl;dr - I trained a tiny Mamba-based autoregressive language model and wrote an inference engine to run it on the 8-bit 6502 processor (from 1975, with 32KB RAM). Running it on my dad's BBC Micro generated the text below. once upon a time tom and lily saw things lily were sad her house he heartd them ilily and tom said yes she saw a little girl smiled tom was so excited her mom said yes The MOS 6502 is an 8-bit microprocessor released in 1975, powering the BBC Micro and the Apple II . I am lucky enough to have access to my dad's BBC Model B from the 80s; I wanted to see, using modern machine learning, what the strongest language model we could fit on this machine was. Unsurprisingly, this poses significant challenges. The model weights and inference code need to be contained within 25KB of user-space memory — my final configuration was 9KB inference code and 13KB model weights. The CPU only operates on an 8-bit integer datatype, and doesn't include multiplication in its instruction set.…