Menu

From 1 Minute 56 Seconds to 3 Seconds: Optimizing Database Inserts with Batch Processing
📰

From 1 Minute 56 Seconds to 3 Seconds: Optimizing Database Inserts with Batch Processing

DEV Community: database·Rishabh·15 days ago
#yBw0eRfC
#devtodatabase#devto#dev#database#batch
Reading 0:00
15s threshold

Sometimes, a feature works perfectly but takes so long that it becomes practically unusable.

Recently, while working on a camera management feature at my office, I encountered exactly this problem.

The Problem

Whenever we assigned cameras to a user, the operation took approximately 1 minute and 56 seconds to complete.

But there was another problem: our server had a request timeout of just 30 seconds.

This meant the API would time out long before the operation could finish.

Clearly, something needed optimization.

Finding the Bottleneck

I started debugging the camera assignment workflow to understand where the time was being consumed.

After tracing the execution, I found the issue.

There was a "for" loop that processed every user-camera mapping individually.

For each mapping, the application was:

Opening a database connection.

Executing an INSERT query.

Closing the database connection.

This process repeated for every camera-user mapping.

When assigning cameras in bulk, this resulted in hundreds…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.