←back to thread

137 points cdesai | 2 comments | | HN request time: 0.421s | source
Show context
jeffrallen ◴[] No.45673746[source]
Tldr: your build system is so f'd that you have gigs of unused source and hundreds of repeated executions of the same build step. They can fix that. Or, you could, I dunno, fix your build?
replies(1): >>45675105 #
jayd16 ◴[] No.45675105[source]
You could just have a mono-repo with a large amount of assets that aren't always relevant to pull.

Incremental builds and diff only pulls are not enough in a modern workflow. You either need to keep a fleet of warm builders or you need to store and sync the previous build state to fresh machines.

Games and I'm sure many other types of apps fall into this category of long builds, large assets, and lots of intermediate build files. You don't even need multiple apps in a repo to hit this problem. There's no simple off the shelf solution.

replies(2): >>45675448 #>>45675763 #
1. Dylan16807 ◴[] No.45675448[source]
For a large amount of sometimes-relevant assets, is mapping them over NFS a bad solution? SourceFS also gets them across the network on demand, right?

And a fleet of warm builders seems pretty reasonable at that scale.

SourceFS sounds useful for extra smart caching but some of these problems do sound like they're just bad fixable configuration.

replies(1): >>45675588 #
2. jayd16 ◴[] No.45675588[source]
> And a fleet of warm builders seems pretty reasonable at that scale.

It's actually pretty hard. The more builders you have the older the workspace gets and scaling up or cycling machines causes the next builds to be super slow. Game engines end up making central intermediate asset caches like Unreal's UBA or Unity's Cache Server.