Menu

1 / 2

What Breaks When You Migrate Petabytes: A Field Guide

DEV Community: aws·Prajakta Shende·about 20 hours ago
#YJkuvcIn
#devtoaws#devto#dev#migration#cutover
Reading 0:00
15s threshold

By Prajakta Shende — Database Customer Engineer at Google Cloud, with close to a decade at AWS in the data space before that.

Every migration plan works on the whiteboard. I've spent a decade moving production databases — petabytes of them, across engines, clouds, and more cutover nights than I care to count — and I can tell you exactly where the plan meets reality. Petabytes don't break migrations in random places. They break them in five predictable ones, in roughly this order.

1. The throughput math nobody does

Before any dates get committed, three numbers decide your migration: how fast the source can be read, how fast the network can move data, and how fast the target can write. Your actual migration speed is the slowest of the three — and it's almost never the one the plan assumed.

The second mistake is treating the bulk copy as the schedule. It isn't. The cutover window is set by change-data-capture lag: how long it takes the target to replay every change that landed during the copy.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.