←back to thread

Use DuckDB-WASM to query TB of data in browser

(lil.law.harvard.edu)
224 points mlissner | 3 comments | 31 Oct 25 17:37 UTC | HN request time: 0.598s | source
Show context
dtech ◴[01 Nov 25 09:40 UTC] No.45780399[source]▶
My company tried DuckDB-WASM + parquet + S3 a few months ago but we ended up stripping it all out and replacing it with a boring REST API.

On paper it seemed like a great fit, but it turned out the WASM build doesn't have feature-parity with the "normal" variant, so things that caused us to pick it like support for parquet compression and lazy loading were not supported. So it ended up not having great performance while introducing a lot of complexity, and also was terrible for first page load time due to needing the large WASM blob. Build pipeline complexity was also inherently higher due to the dependency and data packaging needed.

Just something to be aware of if you're thinking of using it. Our conclusion was that it wasn't worth it for most use cases, which is a shame because it seems like such a cool tech.

replies(2): >>45780605 #>>45780888 #
1. mentalgear ◴[01 Nov 25 10:28 UTC] No.45780605[source]▶
> WASM build doesn't have feature-parity with the "normal" variant

It's a good point, but the wasm docs state that feature-parity isn't there - yet. It could certainly be more detailed, but it seems strange that your company would do all this work without first checking the feature-coverage / specs.

> WebAssembly is basically an additional platform, and there might be platform-specific limitations that make some extensions not able to match their native capabilities or to perform them in a different way.

https://duckdb.org/docs/stable/clients/wasm/extensions

replies(1): >>45780815 #
2. dtech ◴[01 Nov 25 11:19 UTC] No.45780815[source]▶
Note that your docs specifically mentions parquet was supported, but we found out the hard way some specific features turned out not to be supported with WASM + parquet. I did a quick glance at your docs and could not find references to that, so I'm not surprised it was missed.

It was a project that exploited a new opportunity so time-to-market was the most important thing, I'm not suprised these things were missed, and replacing the data loading mechanism was maybe 1 week of work for 1 person, so it wasn't that impactful a change later.

replies(1): >>45781094 #
3. mentalgear ◴[01 Nov 25 12:23 UTC] No.45781094[source]▶
Fair point, thx for sharing your experiences ! You might want to edit the duck-wasm docs in that regard to alert others/the team of this constraint.