Sahil Walia , a senior technical architect at Snowflake, joins host Robert Blumen for a conversation about schema migration and data migration. Together, they discuss the basics of schema: do all data sets have schema?; what drives schema change?; are data migration and schema migration the same thing? Must they happen at the same time? They then talk about the challenges and failure modes of schema migration, including challenges at scale, migrating without downtime, the concept of “safety” in data migrations, and safety patterns for data migration – double writes, read repair, replication, ending on the Apache Iceberg case study.
Brought to you by IEEE Computer Society and IEEE Software magazine .
Show Notes
Related Episodes
SE Radio 736: Sahil Walia on Apache Iceberg
SE Radio 456: Tomer Shiran on Data Lakes
SE Radio 334: David Calavera on Zero-downtime Migrations and Rollbacks with Kubernetes
SE Radio 285: James Cowling on Dropbox’s Distributed Storage System
Other References
Sahil Walia –…