Why deletes reach bronze but never reach silver when every hop runs in one Lakeflow Spark Declarative Pipeline, how I found the cause, and a small change that makes them flow all the way to gold.
A colleague sent me a message on Slack.
They had a multi-hop pipeline: landing, bronze, silver. Every hop used AUTO CDC, and every table had Change Data Feed (CDF) turned on.
Updates worked everywhere. Deletes went from landing to bronze. The bronze change feed even showed the delete.
But silver never deleted the row.
They also found a community thread with the same problem. A Databricks employee answered that it is supported, and listed the usual checks. The author checked all of them on a recent runtime, and it still failed. The thread ends with "it seems to work for single-hop only".
I wanted to understand why, so I built the smallest pipeline that shows the problem.
This guide walks you through what I found.…