←back to thread

Be Aware of the Makefile Effect

(blog.yossarian.net)
431 points thunderbong | 1 comments | 11 Jan 25 04:13 UTC | HN request time: 0.214s | source
Show context
solatic ◴[11 Jan 25 21:15 UTC] No.42668983[source]▶
> Think about CI/CD setups, where users diagnose their copy-pasted CI/CD by doing print-style debugging over the network with a layer of intermediating VM orchestration. Ridiculous!

I don't think the author understands the point of "CI/CD systems". And I don't really blame them, because workload orchestration systems have been abused and marketed to the point where we call them CI/CD systems instead. Sure, if you think the point of CI/CD is to just provide a unified build and deploy it somewhere, you can write that in whatever language you like, and not need to know a bunch of YAML-fu.

But the whole point of workload orchestration systems is to configure the flow of workloads between machines as they inherently execute on different machines. The status quo is to debug over the network because, fundamentally, different machines will be connected by a network and the workload orchestration system is figuring out which machine to put it on.

If you think you can just run your CI/CD on a single server without networking or virtualization, I have some very large, parallelized testing suites to show you.

replies(1): >>42669126 #
semi-extrinsic ◴[11 Jan 25 21:41 UTC] No.42669126[source]▶
> If you think you can just run your CI/CD on a single server without networking or virtualization, I have some very large, parallelized testing suites to show you.

Nowadays you can get a single server with 256 cores and several terabytes of memory. I would be interested to learn what kind of testing suites have actual needs beyond that.

Without virtualization though is definitely no problem. The whole docker/k8s/whatever shtick is mainly because devs think it's more fun to invent new things than to learn how to use old ones properly. At least as long as you're running your own code on your own hardware, there is not a single thing solved by virtualization that wouldn't be solved equally well (or better) with traditional tools like environment modules and Slurm.

replies(2): >>42669515 #>>42671732 #
1. solatic ◴[12 Jan 25 06:40 UTC] No.42671732[source]▶
And you can get desktop workstations with similarly high core counts and RAM. You're missing the point. Your strategy can either depend on whether or not you can afford to buy out larger and larger vertically scaled servers, or you can plan for horizontal scaling. Almost nobody is willing to sign off on the vertical scaling strategy because the sheer presence of a ceiling frightens executives.

And yes, in enterprise, two things are usually at play: you need to test a system where the architecture includes the combined architecture of multiple corporate acquisitions, more than one of which were Vertical Monsters, and more than one of which presumed horizontal scaling; and where deployment scripts must be run from behind a no-ingress-permitted firewall, which means having workload orchestration runners installed behind that firewall.