Menu

Postgres and Iceberg: pg_lake, pg_duckdb and the Postgres lakehouse
πŸ“°

Postgres and Iceberg: pg_lake, pg_duckdb and the Postgres lakehouse

DEV Community: databaseΒ·Mohammed Arshad AnsariΒ·11 days ago
#XtxYB04j
#devtodatabase#devto#dev#postgres#iceberg
Reading 0:00
15s threshold

"Postgres Iceberg" covers four different jobs. You might want to query Iceberg or Parquet files on S3 from Postgres. You might want Postgres to write Iceberg tables. You might want Postgres to be the catalog that other engines use. Or you might want your Postgres data to land in Iceberg for a warehouse to read. Each job has a different tool, and two of the tools people still recommend are no longer maintained.

This post sorts them out. I have not run pg_lake or Iceberg in production. What I did run, on my laptop: PyIceberg 0.12.0 with a plain Postgres 16 container as the Iceberg catalog, and the pg_duckdb Docker image (Postgres 18.1) reading the same table. The numbers below come from those runs.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.