Transactional Object Storage?

Pretty cool! Do you have any ideas already about how to make it work with S3, considering it doesn't support If- headers?

S3 recently added basic matching support (https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3..., https://docs.aws.amazon.com/AmazonS3/latest/userguide/condit...).

They don't have the full suite of GCS's capabilities (https://cloud.google.com/storage/docs/request-preconditions#...) but it's something.

I think it's now much easier to achieve than a year ago. The critical one is conditional writes on new objects, because otherwise you can't safely create transaction logs in the presence of timeouts. This is not enough though.

My approach on S3 would be to ensure to modify the ETag of an object whenever other transactions looking at it must be blocked. This makes it easier to use conditional reads (https://docs.aws.amazon.com/AmazonS3/latest/userguide/condit...) on COPY or GET operations.

For write, I would use PUT on a temporary staging area and then conditional COPY + DELETE afterward. This is certainly slower than GCS, but I think it should work.

Locking without modifying the object is the part that needs some optimization though.