TL;DR

Yes, an inference-only NPU can pretrain a language model, if the training stack is built for it. We pretrained VKORE-325M-NPU-Base (325.6M parameters) from random initialization on the Rebellions ATOM-Max , a commercially available Korean NPU released in 2025 and designed for inference. We trained the identical model on a B200 GPU as a reference. On 1,024 windows of text neither model ever saw, the NPU model scores 0.7993 and the GPU model 0.7994 (cross entropy per byte). Both weights and the evaluation script are public.

Why is "an inference-only NPU cannot train" the default assumption?

Inference accelerators ship with a compiler and runtime built to execute a forward pass of a model trained elsewhere. Pretraining needs much more: a backward pass through every layer, optimizer state, and numerically well-behaved updates for tens of thousands of steps.…