A dashboard does not need a warehouse on day one.
On the HVAC telemetry app I worked on, the analytical surface had to answer questions over a large PyArrow dataset while the product was still small. A warehouse would have meant a second system, a second deployment and a second bill. DuckDB gave us an in-process OLAP engine instead: the API was already Python, the data was already Parquet inside Delta Lake, and the queries needed to be fast, not distributed.
The public proof of concept for that app is here: poc_latency_test . It is a Next.js front end, a FastAPI service and a medallion pipeline. This is what the read path looks like.
The whole trick is not copying anything
DuckDB can query Parquet files directly. The pipeline already writes Delta tables, so the query path starts from the Delta table and hands DuckDB an Arrow dataset.
from deltalake import DeltaTable
def get_delta_dataset ( layer : str , table : str | None = None ): path = ( os . path .…