AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA) platforms provide dedicated hardware for deep learning inference. On NVIDIA Jetson AGX Orin , DLA executes supported neural network operations, allowing applications to offload inference work from the GPU. This can improve power efficiency and leave GPU capacity available for other application tasks.
NVIDIA cuDLA is the runtime API for managing execution on this accelerator. Applications use it to create DLA devices, load compiled networks, register memory buffers, and submit inference tasks. The cuda.bindings.cudla module exposes these APIs directly to Python, so developers can integrate DLA execution into their applications without writing C++ code to manage the runtime.
This post walks through the inference lifecycle on Jetson AGX Orin using an image classification example.…