For use in Golang, I really like ncruces wasm SQLite package - https://github.com/ncruces/go-sqlite3 . Unlike cznic's go package (which is great, btw), the wasm version works well on OpenBSD and the like.
replies(1):
Context: Currently using go-duckdb and while it's working for us, getting rid of cgo would be a huge help. Would be quite interested myself to attempt this.
Wasm is fine for compute (though concurrency is still a somewhat open question).
To have Wasm talk to the outside world, you need “host calls” where the guest calls the host.
On a browser that's Wasm calling JavaScript. On my Go driver, it's Wasm calling Go.
For server side, there's also a standard set of “host calls” modeled around POSIX/Linux syscalls called WASI.
I could've build my project around WASI, but WASI is rather limited (and SQLite support for WASI was more limited even, it's improved a bit since). DuckDB might work out-of-the-box this way.
I, instead, took advantage of SQLite's architecture and replaced its VFS layer with one in Go: https://sqlite.org/vfs.html
So SQLite in Wasm is just doing compute, and I do all the OS level stuff in Go. No need for Wasm concurrency, cause I can load multiple instances of my Wasm which act like independent OS processes that communicate through the filesystem (SQLite excels at this).
As I said, I dunno how well all those decisions would map to DuckDB.
Interesting. So when I am running concurrent readers using your package, it is just loading multiple instances of the wasm code? (I bottleneck to a single writer in the application)
The current approach is not portable to Windows, but it works fine on Linux, macOS, BSD and illumos. In general, portability is hindered more by file locking (I hate POSIX locks) than mmap.
The currently open GitHub issue is more bad default configuration than anything else. Configuring connections to use less memory by default should fix it.
I already have a PR ready for the next release that also opens this up for 32-bit platforms.
Each connection lives in its own isolated sandbox, and only communicates with other connections through the “file system” (which is a virtual abstraction, actually).
WAL mode is the “exception”: a few pages of the sandbox's memory are mapped to a file, and shared by all connections to the same database.
Each sandbox is single threaded, and mostly lock free, does all its business in the calling goroutine, and regularly checks back with the Go runtime to play nice with the Go scheduler.
It's a bit like an OS running multiple processes, with the VFS layer handling all syscalls.