Menu

#Decoding

15 posts

Feed·
14 of 15 posts
Tech & AI Synthesis: BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design
📰
0Read More

Tech & AI Synthesis: BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design

DEV Community: deeplearning·Sharath·about 13 hours ago
#rKsM7kdE
#devtodeeplearning#education#devto#dev#cache

The key-value (KV) cache dominates memory capacity and bandwidth in long-context LLM serving. While 4-bit microscaling formats (NVFP4, MXFP4) shrink the cache and run attention on FP4 Tensor Cores, a fundamental gap...

15s
What would you require before calling an inference change “quality-neutral”? [D]
📰
0Read More

What would you require before calling an inference change “quality-neutral”? [D]

Two GSM8K scores: 96.89% → 96.82%, across all 1,319 test questions. These are from NVIDIA’s Ling-3.0-flash comparison on one DGX Spark. The optimized configuration changes the MoE backend, adds an FP8 output head and enables NEXTN speculative decoding…

15s
Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding
🖼️
0Read More

Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding

NVIDIA Technical Blog·Amr Elmeleegy·25 days ago
#ZsD6f7s2
#developer#dflash#nvidia#model#decoding

As AI systems move from single-turn interactions to coordinated multiagent workflows, low-latency inference becomes increasingly important.

15s
Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson
🖼️
0Read More

Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson

NVIDIA Technical Blog·Elizabeth Goodman·28 days ago
#T5Hat12w
#developer#model#jetson#models#speculative

Running reasoning and agentic AI at the edge has been harder than it needs to be. Until recently, models capable of multi-step reasoning were too large to run…

15s
Release TorchCodec 0.14: HDR Video Decoding for CPU & CUDA, and Fast Wav Decoder · meta-pytorch/torchcodec
📰
0Read More

Release TorchCodec 0.14: HDR Video Decoding for CPU & CUDA, and Fast Wav Decoder · meta-pytorch/torchcodec

Hacker News·5 months ago
#hXmeSSO8
#github#decoding#decoder#torchcodec#float32

TorchCodec 0.14 is out! It is compatible with torch >= 2.11. It comes with two major additions: a fast audio WavDecoder, and support for HDR video decoding! Fast wav decoder TorchCodec now has a de...

15s
Nvidia releases open AI models for quantum computing tasks — 'Ising' said to be 2.5x faster and 3x more accurate than existing tools for decoding
🖼️
0Read More

Nvidia releases open AI models for quantum computing tasks — 'Ising' said to be 2.5x faster and 3x more accurate than existing tools for decoding

Tom's Hardware·Luke James·7 months ago
#uiuWgRpt

From Latest from Tom's Hardware: Nvidia releases open AI models for quantum computing tasks — 'Ising' said to be 2.5x faster and 3x more accurate than existing tools for decoding

15s