Menu

1 / 48

A primer on key AI LLM concepts — KV cache, weights, quantization, and how inference actually works

Reading 0:00
15s threshold

As hardware for running local AI models continues to move to the forefront, such as Nvidia's RTX Spark, a new set of acronyms, definitions, and terminology is popping up. Given the pace of releases, we've put together a list of AI's new nomenclature to familiarize you with new concepts and pieces of terminology that you'll likely see in Tom's Hardware Premium articles.

This primer should get you started on the world of AI and be handy the next time news on the topic comes around, and possibly when you're downloading some models to try out and tweak for yourself. For the context of this article, we're going to stick with notions related to text LLMs, or in common parlance, chat, programming, and most agentic bots.