←back to thread

SourceFS: A 2h+ Android build becomes a 15m task with a virtual filesystem

(www.source.dev)
137 points cdesai | 2 comments | 22 Oct 25 12:39 UTC | HN request time: 0s | source
Show context
Ericson2314 ◴[22 Oct 25 15:46 UTC] No.45670912[source]▶
The headline times are a bit ridiculous. Are they trying to turn https://github.com/facebook/sapling/blob/main/eden/fs/docs/O... or some git fuse thing into a product?
replies(2): >>45671058 #>>45671060 #
zokier ◴[22 Oct 25 15:55 UTC] No.45671060[source]▶
Well they also claim to be able to cache build steps somehow build-system independently.

> As the build runs, any step that exactly matches a prior record is skipped and the results are automatically reused

> SourceFS delivers the performance gains of modern build systems like Bazel or Buck2 – while also accelerating checkouts – all without requiring any migration.

Which sounds way too good to be true.

replies(4): >>45671827 #>>45671987 #>>45672236 #>>45677349 #
1. fukka42 ◴[22 Oct 25 16:47 UTC] No.45671827[source]▶
Seems viable if you can wrap each build stap with a start/stop signal.

At the start snapshot the filesystem. Record all files read & written during the step.

Then when this step runs again with the same inputs you can apply the diff from last time.

Some magic to automatically hook into processes and doing this automatically seems possible.

replies(1): >>45674101 #
2. bananaquant ◴[22 Oct 25 19:37 UTC] No.45674101[source]▶
I think I got the magic part. You can store all build system binaries in the VFS itself. When any binary gets executed, VFS can return a small sham binary instead that just checks command line arguments, if they match, checks the inputs, and if they match, applies the previous output. If there is any mismatch, it can execute the original binary as usual and make the new output. Easy and no process hacking necessary.