←back to thread

Pg_parquet: An extension to connect Postgres and parquet

(www.crunchydata.com)
85 points craigkerstiens | 2 comments | 17 Oct 24 16:18 UTC | HN request time: 0s | source
Show context
linuxhansl ◴[17 Oct 24 20:55 UTC] No.41873697[source]▶
Parquet itself is actually not that interesting. It should be able to read (and even write) Iceberg tables.

Also, how does it compare to pg_duckdb (which adds DuckDB execution to Postgres including reading parquet and Iceberg), or duck_fdw (which wraps a DuckDB database, which can be in memory and only pass-through Iceberg/Parquet tables)?

replies(2): >>41874044 #>>41874177 #
1. AdamProut ◴[17 Oct 24 21:33 UTC] No.41874044[source]▶
Had a similar thought. Azure Postgres has something similar to pg_parquet (pg_azure_storage), but we're looking into replacing it with pg_duckdb assuming the extension continues to mature.

It would be great if the Postgres community could get behind one good opensource extension for the various columnstore data use cases (querying data stored in an open columnstore format - delta, iceberg, etc. being one of them). pg_duckdb seems to have the best chance at being the goto extension for this.

replies(1): >>41874183 #
2. mslot ◴[17 Oct 24 21:51 UTC] No.41874183[source]▶
Fun fact, I created pg_azure_storage :)