Menu

1 / 3

Accelerating Reinforcement Learning Weight Refit with the Reshard Collective

NVIDIA Technical Blog·Tanya Lenz·9 days ago
#MGFUI4nP
#developer#refit#reshard#gpus#training
Reading 0:00
15s threshold

During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with the current policy model. For training efficiency, it is common to run inference and training in parallel on separate GPU resources in a disaggregated setup. Although the policy model is updated only on the training GPUs, the system must periodically refresh the weights on the inference GPUs with the latest parameters generated on the training GPUs. This transfer of model weights between training and inference GPUs is called refit .

Refit involves two defining characteristics that pose challenges.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.