←back to thread

Use DuckDB-WASM to query TB of data in browser

(lil.law.harvard.edu)
224 points mlissner | 2 comments | 31 Oct 25 17:37 UTC | HN request time: 0.494s | source
Show context
leetrout ◴[31 Oct 25 23:39 UTC] No.45777905[source]▶
I built something on top of DuckDB last year but it never got deployed. They wanted to trust Postgres.

I didn't use the in browser WASM but I did expose an api endpoint that passed data exploration queries directly to the backend like a knock off of what new relic does. I also use that same endpoint for all the graphs and metrics in the UI.

DuckDB is phenomenal tech and I love to use it with data ponds instead of data lakes although it is very capable of large sets as well.

replies(2): >>45777957 #>>45781403 #
1. whalesalad ◴[31 Oct 25 23:48 UTC] No.45777957[source]▶
Cool thing about DuckDB is it can be embedded. We have a data pipeline that produces a duckdb file and puts it on S3. The app periodically checks that assets etag and pulls it down when it changes. Most of our DB interactions use PSQL, but we have one module that leverages DuckDB and this file for reads. So it's definitely not all-or-nothing.
replies(1): >>45779000 #
2. zenmac ◴[01 Nov 25 03:20 UTC] No.45779000[source]▶
Are you using pg_duckdb to embedded it inside postgres and access it via psql or other pg clients?