Menu

When long-running requests exhaust the connection pool and why standard HPA won’t help
📰

When long-running requests exhaust the connection pool and why standard HPA won’t help

DEV Community: kubernetes·Denis Toropov·9 days ago
#TWd1JYTk
#devtokubernetes#devto#dev#database#pool
Reading 0:00
15s threshold

Consider a service running with two pods. It has an HPA configured to add a pod when CPU utilization reaches 80%. This helps the application scale as CPU demand grows. But HPA does not choose a pod for each individual request: a request can be routed to any available replica.

The problem appears when several users start a long-running request at almost the same time and all their requests land on the same pod. For example, users may run reports that hold a database connection for several minutes. Kubernetes does not account for how many HikariCP connections are occupied in each pod, so requests can all reach one pod and exhaust its pool—even while the other pod still has free connections.

If each pod’s pool is limited to five connections, five reports running at the same time can occupy every connection in one replica. New requests to that pod will wait for a connection to become available. Depending on the application’s probes and failure handling, the pod may also become unhealthy or overloaded.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.