←back to thread

S3 as a Git remote and LFS server

(github.com)
194 points kbumsik | 4 comments | 19 Oct 24 10:37 UTC | HN request time: 0s | source
Show context
philsnow ◴[19 Oct 24 19:13 UTC] No.41889926[source]▶
I'm surprised they just punt on concurrent updates [0] instead of locking with something like dynamodb, like terraform does.

[0] https://github.com/awslabs/git-remote-s3?tab=readme-ov-file#...

replies(3): >>41890047 #>>41890892 #>>41890916 #
1. ncruces ◴[19 Oct 24 21:24 UTC] No.41890892[source]▶
Google Cloud Storage is good enough to implement locks all by itself: https://reddit.com/r/golang/comments/t52d4f/gmutex_a_global_...

Doesn't S3 provide primitives to do the same? At least since moving to strong read-after-write consistency?

PS: I wrote the above package. Happy to answer questions about it.

replies(1): >>41892398 #
2. kbumsik ◴[20 Oct 24 02:17 UTC] No.41892398[source]▶
Conditional write is just added to S3 2 month ago: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3...
replies(1): >>41898472 #
3. laurencerowe ◴[20 Oct 24 21:07 UTC] No.41898472[source]▶
Unfortunately this functionality is much more limited in S3 as you can only use `If-None-Match: *` to prevent overwrites. https://docs.aws.amazon.com/AmazonS3/latest/userguide/condit...

GCS also allows for conditional overwrites using `If-Match: <etag>` which means you can do optimistic concurrency control. https://cloud.google.com/storage/docs/request-preconditions

replies(1): >>41902206 #
4. ncruces ◴[21 Oct 24 09:23 UTC] No.41902206{3}[source]▶
Yeah, it might still be possible to implement a mutex based on just the existence of an object, but it'll be harder to add expiration/liveness which I find essential.