#Vllm

Contribute to ryanzhou/deepseek-v4-flash-mi300x development by creating an account on GitHub.

LocalAI is the open source AI engine. Run any model, LLMs, vision, voice, image and video, on any hardware. No GPU required.

Contribute to kyuz0/amd-strix-halo-vllm-toolboxes development by creating an account on GitHub.

A deep dive into memory fragmentation, paged memory management, and why PagedAttention can deliver up...

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag. - huawei-csl/KVarN

Learn how to self-host Langfuse, connect it to vLLM, and build full LLM observability with traces, tokens, latency, and dashboards — from scratch.

From Dev.to - tutorial: How to Deploy Mistral Nemo with vLLM + Flash Attention on a $12/Month DigitalOcean GPU Droplet: 3x Faster Inference at 1/95th Claude Cost

From Dev.to - ai: How to Deploy Llama 3.2 with vLLM + Batch Processing on a $8/Month DigitalOcean Droplet: Asynchronous Inference at 1/125th Claude Cost

Inside vLLM's CPU backend: a new contributor's notes Most of the public technical writing...

Deploy LLMs in production with vLLM, Hugging Face TGI, and ONNX Runtime. Learn quantization techniques, GPU memory optimization, and serving strategies.

From Dev.to - tutorial: How to Deploy Llama 3.2 90B with vLLM + Speculative Decoding on a $16/Month DigitalOcean GPU Droplet: 2.5x Faster Inference at 1/110th Claude Cost

From Dev.to - webdev: How to Deploy Llama 3.2 70B with vLLM + Quantization on a $12/Month DigitalOcean GPU Droplet: Enterprise Inference at 1/110th Claude Cost

From Dev.to - machinelearning: Counterintuitive: WSL2 + vllm cannot fit Qwen2.5-7B-1M on 6GB VRAM where Windows transformers can

From Dev.to - webdev: How to Deploy Mistral Large with vLLM on a $20/Month DigitalOcean GPU Droplet: Enterprise Inference at 1/80th Claude Cost

ZAYA1-8B packs 760M active parameters into an 8.4B MoE that beats DeepSeek-R1 on AIME 2025. Here is what developers need to know.

From Dev.to - tutorial: How to Deploy Llama 3.2 405B with vLLM on a $48/Month DigitalOcean GPU Droplet: Frontier-Grade Reasoning at 1/120th Claude Opus Cost

From Dev.to - ai: How to Deploy Qwen2.5 72B with vLLM + FastAPI on a $20/Month DigitalOcean GPU Droplet: Production Inference at 1/90th Claude Cost

Comparing SGLang vs vLLM? See how they differ on serving architecture, runtime features, deployment fit, and production GPU infrastructure.