If you really embrace "local first" just use the file system, and the user can choose from many solutions like git, box, etc.
I hate signing up for your sync just as much as any other SAAS, but it's even more opaque and likely to break.
If you really embrace "local first" just use the file system, and the user can choose from many solutions like git, box, etc.
I hate signing up for your sync just as much as any other SAAS, but it's even more opaque and likely to break.
First is that yeah, local first, but I also want concurrency. If it's just local first, you're right, any old sync will do. But I want more than that. I want to not have to think (a la dropbox, being slick). I want my wife and I to be able to make separate edits on our phones when we're in a dead zone.
Second is that sync works a lot better when it has deep knowledge of the data structure and semantics. Git and box both have significant shortcomings, but both exacerbated by the concurrency desire.
If files are insufficient, what data-structure would make modular sync possible for multiple applications in an OS?
And I’m not suggesting one doesn’t exist, I’m challenging to present a comprehensive solution, that probably involved operating systems.
> I want my wife and I to be able to make separate edits on our phones when we're in a dead zone.
Files do this.
I think it boils down to provenance and concurrency. If we edit the same line a file, that's ba merge conflict when it really should be simple and something I shouldn't have to bother with. And when we do do the same line edit, I'd love to have provenance on that data.
Granted, those aren't local first thing exactly, but I think there will be apps that want all of that.
Note that this does not even need two users: I hit this problem with a desktop and laptop and self-hosted NextCloud myself.
In general, a filesystem that actually stored both raw data (to fail-over to), but also a per-format event log, and maybe even app specific events (imagine a PNG changes, we could have any change recorded as raw bytes, generic bitmap image operation like "modify pixels at x,y to ..." and app-specific log like "gimp: apply sharpen filter on polygon area ...").
This would allow the other side to attempt to do the smartest sync it has (if it has a compatible version of gimp, it could decide to apply the filter, otherwise fall back to raw pixel changes if no conflicts, and then fall back to full file contents reconciliation).
Just like MIME handlers get registered, if file systems provided such change logs, some could have very advanced sync systems with this support from "filesystems".
I’m also suspect of logs as a general form of conflict resolution as you are just hoping the two edits don’t touch the same area. And if they do then you are left in an invalid state.
You brought up zips. Pile of files seems like a way you can divide up data so it can have more pieces that are mergable/diffable.
For example “the folder can contain N files” or “there must be exactly 1 of this file”.
For zip and other container-type files, you'd have log entries to the tune of "changed contained file foo.png: ...".
Operating systems would need to support some basic operations: container file operations like zip files, basic bitmap image editing, basic text document diffing, structured text diffing (XML, JSON, YAML...), etc...
Applications would provide OS-registered services (like MIME handlers are registered today) that can interpret and produce even more semantic events on top of the existing ones.
The environment could offer an interface during "syncing" when it detects a conflict to resolve it using one of the generic (or not) conflict resolution mechanisms (use local or remote version completely; use incremental delta if there is some generic semantic diff; app-provided capability if present on both sides).
Now, you are right that this can be implemented with this log being a regular file next to the file itself and completely user-space, but you will hit issues if you are not able to tie it nicely to things like fwrite/fsync and similar syscalls.
Obviously, for it to make sense, it needs to be widely accepted as the approach, which is what the local-first movement is trying to achieve with CRDTs.