←back to thread

Use DuckDB-WASM to query TB of data in browser

(lil.law.harvard.edu)
224 points mlissner | 1 comments | 31 Oct 25 17:37 UTC | HN request time: 0.202s | source
Show context
mlissner ◴[31 Oct 25 17:37 UTC] No.45774572[source]▶
OK, this is really neat: - S3 is really cheap static storage for files. - DuckDB is a database that uses S3 for its storage. - WASM lets you run binary (non-JS) code in your browser. - DuckDB-Wasm allows you to run a database in your browser.

Put all of that together, and you get a website that queries S3 with no backend at all. Amazing.

replies(4): >>45775637 #>>45775690 #>>45775706 #>>45780371 #
timeflex ◴[31 Oct 25 19:15 UTC] No.45775637[source]▶
S3 might be relatively cheap for storing files, but with bandwidth you could easily be paying $230/mo. If you make it public facing & want to try to use their cloud reporting, metrics, etc. to prevent people for running up your bandwidth, your "really cheap" static hosting could easily cost you more than $500/mo.
replies(3): >>45775880 #>>45777187 #>>45777948 #
theultdev ◴[31 Oct 25 19:39 UTC] No.45775880[source]▶
R2 is S3 compatible with no egress fees.

Cloudflare actually has built in iceberg support for R2 buckets. It's quite nice.

Combine that with their pipelines it's a simple http request to ingest, then just point duckdb to the iceberg enabled R2 bucket to analyze.

replies(3): >>45776489 #>>45777933 #>>45778926 #
1. greatNespresso ◴[31 Oct 25 20:44 UTC] No.45776489[source]▶
Was about to jump in to say the same thing. R2 is a much cheaper alternative to S3 that just works and I have used it with DuckDB, works smoothly