Menu

Making cudaMemcpy 5x faster on a B300 [P]

Reading 0:00
15s threshold
Making cudaMemcpy 5x faster on a B300 [P]

https://preview.redd.it/cjz9xy3rht1i1.png?width=1600&format=png&auto=webp&s=0eea8a5d9e53d5ea4593ce1a629bc7342a049ffa

Hi everyone,

I made a tool that can speed up pageable transfers for cudaMemcpy. For example:

# B300, copying a 1 GB tensor to the GPU and back: import time, torch x = torch.ones(1 << 30, dtype=torch.uint8) y = x.to("cuda"); y.cpu() for name, copy in [("CPU -> GPU", lambda: (x.to("cuda"), torch.cuda.synchronize()))]: t = time.perf_counter() for _ in range(5): copy() print(f"{name}: {5 * x.nbytes / (time.perf_counter() - t) / 1e9:.1f} GB/s") $ python bench.py CPU -> GPU: 10.2 GB/s $ cmt tune -- python bench.py CPU -> GPU: 51.5 GB/s 

blog: https://maknee.github.io/blog/2026/CudaMemcpy/

code: https://github.com/Maknee/cudamemcpy-tuner

submitted by /u/makneeee
[link] [comments]