I pull the data every hour with a GitHub action and redeploy the site nightly. The duckdb is regenerated every time and I can flexibly change its structure as required. So far I didn’t change much (eg. no normalization) since it’s not really needed. It also turns out to be surprisingly small, with compression a few megabytes. I haven’t written much SQL in some time and duckdb is very powerful.
Most of the texts are AI generated as I’m usually very bad coming up with this. I really gotta learn that one day. Webdesign is the other weakness, but with all these UI libraries these days it’s less of a problem.
I guess it’s pretty niche, but maybe interesting for some.
It’s a fun little project.