←back to thread

172 points frectonz | 5 comments | | HN request time: 0.874s | source

pglite-fusion is a PostgreSQL extension that allows you to embed SQLite databases into your PostgreSQL tables by enabling the creation of columns with the `SQLITE` type. This means every row in the table can have an embedded SQLite database.

In addition to the PostgreSQL `SQLITE` type, pglite-fusion provides the `query_sqlite`` function for querying SQLite databases and the `execute_sqlite` function for updating them. Additional functions are listed in the project’s README.

The pglite-fusion extension is written in Rust using the pgrx framework [1].

----

Implementation Details

The PostgreSQL `SQLITE` type is stored as a CBOR-encoded `Vec<u8>`. When a query is made, this `Vec<u8>` is written to a random file in the `/tmp` directory. SQLite then loads the file, performs the query, and returns the result as a table containing a single row with an array of JSON-encoded values.

The `execute_sqlite` function follows a similar process. However, instead of returning query results, it returns the contents of the SQLite file (stored in `/tmp`) as a new `SQLITE` instance.

[1] https://github.com/pgcentralfoundation/pgrx

1. simonw ◴[] No.42183335[source]
They /tmp file mechanism sounds like a bit of a hack, is that definitely necessary?

It may be possible to create a SQLite in-memory database instead and then load the binary blob data into it using the backup API or some kind of trick with VACUUM INTO.

replies(2): >>42183582 #>>42184303 #
2. michelpp ◴[] No.42183582[source]
I think the right approach would be to store the sqlite database as a varlena type that can be TOASTed and then "expanded" using the Expanded Datum API so that it's a live open database connection for the life of the transaction:

https://www.postgresql.org/docs/17/xtypes.html#XTYPES-TOAST

https://github.com/postgres/postgres/blob/master/src/include...

replies(1): >>42183615 #
3. frectonz ◴[] No.42183615[source]
Thanks i will look into this more, /tmp stuff is most definitely a hack.
replies(1): >>42186062 #
4. ◴[] No.42184303[source]
5. michelpp ◴[] No.42186062{3}[source]
Here's an example of a simple expanded object to start from:

https://github.com/michelp/pgexpanded