←back to thread

245 points gatesn | 1 comments | | HN request time: 0s | source
Show context
Havoc ◴[] No.41840621[source]
Can one edit it in place?

That’s the main thing currently irritating me about parquet

replies(2): >>41841001 #>>41845342 #
aduffy ◴[] No.41841001[source]
You're unlikely to find this with any analytic file format (including Vortex). The main reason is that OLAP systems generally assume an immutable distributed object/block layer (S3, HDFS, ABFS, etc.).

It's then generally up to a higher-level component called a table format to handle the idea of edits. See for example how Apache Iceberg handles deletes https://iceberg.apache.org/spec/#row-level-deletes

replies(2): >>41841714 #>>41848589 #
1. Havoc ◴[] No.41848589[source]
I see. Hadn’t made the connection to S3 etc. that makes sense though. Thanks for explaining