Benchmarking Dify RAG Pipelines: Token Consumption, Cache Hit Rates, and Retrieval Latency Under Load

When evaluating Dify for production RAG workloads, the critical questions are straightforward: how much does each query actually cost in tokens, what happens to latency under concurrent load, and at what scale does prompt caching deliver measurable savings? This benchmark measures Dify's document retrieval and context assembly pipeline under realistic multi-user traffic.

Test Topology

The harness uses a self-hosted Dify instance (v0.12.3) running on an 8-core VPS with 16GB RAM and Postgres 15 for vector storage. Documents indexed: 2,400 markdown files totaling ~18M tokens, chunked at 512 tokens with 15% overlap.…