Menu

Large Uploaded Training Files: 3 Signals for Database Blobs or Object Storage
πŸ“°

Large Uploaded Training Files: 3 Signals for Database Blobs or Object Storage

DEV Community: postgresΒ·HoratioFox1281Β·2 days ago
#c8JNYRbC
#devtopostgresql#devto#dev#object#upload
Reading 0:00
15s threshold

Pick object storage for the bytes and keep one row per file in Postgres. For a Node.js backend that ingests large user-uploaded documents β€” lecture masters, SCORM packages, graded submission exports, the training artifacts an edtech platform piles up every cohort β€” that split survives contact with production far better than pushing blobs through the database. Large files in your primary database degrade backups, WAL volume and failover time in roughly that order, and all three degrade quietly.

Three measurements decide it. Not opinions.

The version of this choice worth trusting is the one you can put on a dashboard: the p99 size of an incoming artifact, the sustained write throughput of the upload path, and how long the retention clock runs before a deletion has to be provable. Measure those first, then read the table below.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.