SourceFS: A 2h+ Android build becomes a 15m task with a virtual filesystem

The headline times are a bit ridiculous. Are they trying to turn https://github.com/facebook/sapling/blob/main/eden/fs/docs/O... or some git fuse thing into a product?

Well they also claim to be able to cache build steps somehow build-system independently.

> As the build runs, any step that exactly matches a prior record is skipped and the results are automatically reused

> SourceFS delivers the performance gains of modern build systems like Bazel or Buck2 – while also accelerating checkouts – all without requiring any migration.

Which sounds way too good to be true.

Yeah, I agree. This part is hand waved away without any technical description of how they manage to pull this off since knowing what is even a build step and what dependencies and outputs are are only possible at the process level (to disambiguate multi threaded builds). And then there’s build steps that have side effects which come up a lot with CMake+ninja.

A fuse filesystem can get information about the thread performing the file access: https://man.openbsd.org/fuse_get_context.3

So they could in principle get a full list of dependencies of each build step. Though I'm not sure how they would skip those steps without having an interposer in the build system to shortcut it.