Menu

Databricks Lakebase sinks: streaming straight into Postgres

#sql#redditsql#when#lakebase#databricks
Reading 0:00
15s threshold

I'm a Solutions Architect at Databricks, and I've been spending a lot of time on the Lakebase sink. It lets a Structured Streaming query write directly into a Lakebase (Postgres) table with batching, automatic retries, and workspace-managed auth. You don't need a custom foreach writer or your own connection pooling. I have many customers that have to manage both the foreach writer and connection pooling and it distracts from the job at at hand, serving data to front end applications for real time updates (without the overhead). Having a databse next to your lakehouse acclerates use cases and value for your business. And if your are not limited to just Lakebase, any postgres database is supported! Go check it out today if you haven't already: https://docs.databricks.com/aws/en/structured-streaming/lakebase#when-to-use-the-lakebase-sink

submitted by /u/Wise_Ear_4064
[link] [comments]