GRPO (Group Relative Policy Optimization) is a reinforcement learning method that scores several answers to the same prompt and moves the model toward the ones that beat their group's average, so it needs no critic model. Nodus runs it as a managed runtime, nodus/grpo-lora , which trains a LoRA adapter, by default on one GPU, grades every answer on the platform and caps the run at a dollar limit you set. With QLoRA, an 8B model needs only about 4.5 GB for weights and trainable state, and the Nodus preset runs it on a single GPU with 48 GB or more.

Where does GRPO come from?

GRPO was introduced in the DeepSeekMath paper as "a variant of Proximal Policy Optimization (PPO)" that also cuts PPO's memory use. In that paper it lifted DeepSeekMath-Instruct 7B from 82.9% to 88.2% on GSM8K and from 46.8% to 51.7% on MATH.

It became widely known a year later.…