←back to thread

Use DuckDB-WASM to query TB of data in browser

(lil.law.harvard.edu)
224 points mlissner | 1 comments | 31 Oct 25 17:37 UTC | HN request time: 0s | source
Show context
wewewedxfgdf ◴[31 Oct 25 19:33 UTC] No.45775817[source]▶
I tried DuckDB - liked it a lot - was ready to go further.

But found it to be a real hassle to help it understand the right number of threads and the amount of memory to use.

This led to lots of crashes. If you look at the projects github issues you will see many OOM out of memory errors.

And then there was some indexed bug that crashed seemingly unrelated to memory.

Life is too short for crashy database software so I reluctantly dropped it. I was disappointed because it was exactly what I was looking for.

replies(4): >>45776001 #>>45776020 #>>45776900 #>>45777350 #
mritchie712 ◴[31 Oct 25 21:28 UTC] No.45776900[source]▶
what did you use instead? if you hit OOM with the dataset in duckdb, I'd think you'd hit the OOM with most other things on the same machine.
replies(1): >>45776924 #
1. wewewedxfgdf ◴[31 Oct 25 21:30 UTC] No.45776924[source]▶
The software should manage its own memory not require the developer to set specific memory thresholds. Sure, a good thing to be able to say "use no more than X RAM".