Test Topology

When integrating Dify into a multi-tenant agent platform serving 500+ concurrent users, vector retrieval latency becomes the bottleneck. I ran a comparative stress test between Dify's default pgvector setup and a self-hosted Qdrant cluster to measure real-world performance under sustained load.

Harness:

Load generator: wrk (HTTP) + pytest-asyncio (Python SDK)

Test duration: 300s sustained load

Concurrency: 500 parallel agent sessions

Dataset: 50k embedded documents (1536-dim OpenAI embeddings)

Query pattern: Hybrid search with metadata filters (user_id, timestamp range)

Environment:

Dify v0.6.13, deployed via Docker Compose

PostgreSQL 15 + pgvector 0.5.1 (HNSW index, m=16, ef_construction=64)

Qdrant 1.7.4 (HNSW, m=16, ef_construct=100)

Hardware: 8-core AMD EPYC, 32GB RAM, NVMe SSD

Metric Comparison

Backend Throughput (RPS) p95 Latency (ms) p99 Latency (ms) Memory Overhead (GB) Recall@10

pgvector 487 340 680 4.2 0.91

Qdrant 512 180 290 6.8 0.94

Key Observations:

p99 latency…