Menu

The fastest way to delete 95% of a database is to not delete it | Slicing Prod Data
πŸ“°

The fastest way to delete 95% of a database is to not delete it | Slicing Prod Data

DEV Community: djangoΒ·Pedro BeethovenΒ·about 23 hours ago
#fIKtnj6k
#devtodjango#devto#dev#table#foreign
Reading 0:00
15s threshold

As described here , the initial try of creating a slice of production data was made by using the Django ORM, and it did not work, because even deleting in batches the job was still running after more than 2 hours and the pipeline timed out before finishing it. After taking a closer look at it, deleting is the expensive direction in both Django, where the cascade is resolved in Python by the Collector (as described in the first post), and Postgres (everything goes through the WAL , and the deleted rows are still there until a VACUUM ), and also, its cost grows with the rows that we throw away, which are the majority of the table.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.