Menu

1 / 2

How do you fine-tune Qwen with LoRA in the cloud cheaply?

DEV Community: machinelearning·Viswaretas Kotra·3 days ago
#7z94UQjk
Reading 0:00
15s threshold

Fine-tune Qwen cheaply with one LoRA adapter on the smallest single GPU that holds twice the frozen weights: 16-bit LoRA up to about 9B, 4-bit QLoRA for 14B to 32B dense models. Nodus ships single-GPU Qwen3 presets from 0.6B to 8B in its managed nodus/sft runtime, on cards such as the 48 GB RTX A6000 listed at $0.333/hr on the Nodus pricing page .

Which Qwen models are current, and which are dense or MoE?

Two architecture lines are current, and they train differently. Qwen3 is text-only, with dense models from 0.6B to 32B plus mixture-of-experts (MoE) models such as 30B-A3B. Qwen3.5, Qwen3.6 and Qwen3.8 are vision-language models on a newer hybrid architecture, per the Qwen organization on Hugging Face .

The newest dense model is Qwen3.8-27B . Its model card describes 64 layers in a repeating block of three Gated DeltaNet (linear attention) layers and one gated full-attention layer, and a 262,144-token native context.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.