Be Aware of the Makefile Effect

At my work I've noticed another contributing factor: tools/systems that devs need to interact with at some point, but otherwise provide little perceived value to learn day-to-day.

Example is build system and CI configuration. We absolutely need these but devs don't think they should be expected to deal with them day to day. CI is perceived as a system that should be "set and forget", like yeah we need it but really I have to learn all this just to build the app? Devs expect it to "just work" and if there are complexities then another team (AKA my role) deals with that. As a result, any time devs interact with the system, there's a high motivation to copy from the last working setup and move on with their day to the "real" work.

The best solution I see is meet the devs halfway. Provide them with tooling that is appropriate simple/complex for the task, provide documentation, minimise belief in "magic". Tools like Make kinda fail here because they are too complex and black-box-like.

The office coffee machine is not „set and forget”, but you wouldn’t expect the entire responsibility for it’s maintenance to be evenly distributed between all people that use it. Similarly, CI needs ownership and having it fall on the last developer that attempted to use it is not an efficient way of working.

For me the big problems with CI setups tend to be:

- They're often slow

- They're often proprietary

- They're often dealing with secrets which limits who can work on them

- You generally can't run them locally

So the feedback cycle for working on them is incredibly long. And working on them is therefore a massive pain.

> You generally can't run them locally

I recognize that this is such a disincentive for me taking the initiative to fiddle with and learn about anything like this

Yeah, I think this is the real issue. Too many different tool types that need to interact, so you don't get a chance to get deep knowledge in any of them. If only every piece of software/CI/build/webapp/phone-app/OS was fully implemented in GNU make ;-) There's a tension between using the best tool for the job vs adding yet another tool/dependency.

The local part is my big problem too. I used azure Dev ops in work. I find clicking through the UI to be a miserable experience, Id love to have it running locally so I could view inputs and outputs on the file system. Also yaml is an awful choice, no one I know enjoys working with it. The white space issues just get worse and worse longer your files get.

If you think `make` is “too complex and black-box-like” then you haven't seen `cmake`.

> You generally can't run them locally

GitLab CI gives you local runners. You can completely self-host CI.

Well, yes, but aren't those runners have different configuration than the runners that are actually deployed and used by your company's CI/CD?

Same goes for anything "enterprisey". Last time I set up a big project, I made a commitment that "we should be able to check out and build this whole thing, for as long as humanly possible".

If you think cmake is a good example of more complex than make, then you haven't seen automake/autoconf. The first thing I thought of. You can find tons of tons of configure scripts that check if you're running ancient versions of Unix, checks that a byte is 8 bits wide, and a ton of other pointless checks. They don't do anything with all that information, don't think for a moment that you can actually build the app on Irix, but the checks for it have been passed along for decades likes junk DNA.

Strong agree. The best workflow I've seen uses CICD as a very thin wrapper around in-tree scripts or make files.

If a Dev can run some/all of the "cicd" stuff locally, they can see, control, and understand it. It helps tremendously to have a sense of ownership and calm, vs "cicd is something else, la la la".

(This doesn't always work. We had a team of two devs, who had thin-wrapper CICD, who pretended it was an alien process and refused to touch it. Weird.)

Wat? Our company's CI is GitLab CI, self-hosted. What other runners would we need?

Firstly, automake/autoconf is not `make`, but a different piece of software, and secondly, that you know all those details about it is because it is not black-box-like.

It's not self hosting. It's can I run the build from my local command line and get the same results.

How is running those runners on our own hosts (locally) not self-hosting?

The local ones, mentioned in the original comment that we've been discussing in this thread? The local runner, executing on the developer's machine.

I never said it was. It's a script to generate a script to generate a makefile, more or less.

If it wasn't black box like, why do people keep blindly copying tests which check things which haven't been relevant for decades and in any case would require a ton of manual effort to actually use for something?

Of course it makes no difference whether that runner runs on our local CI host or on my dev machine.

Spoken like someone who has not tried what you are describing. There are two moving parts to your response: a locally hosted runner awaits jobs from GitLab itself, which doesn't help running _locally_, and the other part is that -- back when it existed! -- trying $(gitlab-runner exec) was not a full fledged implementation of the GitLab CI concepts, making it the uncanny valley of "run something locally."

However, as of v16 there is no more exec https://gitlab.com/gitlab-org/gitlab/-/issues/385235 which I guess is good and bad. Good in that it not longer sets improper expectations that it could have plausibly done anything, and bad in that now it joins GitHub Actions[1] in not having any _local_ test strategy aside from "boot up gitlab/gitlab-ce && echo good luck"

1: yes, I'm acutely aware of the 3(?) implementations/forks of nektos/act that claim to do GHA but, again, readme is not software and I can tell you with the utmost certainty they do not do as advertised

+1. The only CI tool that I've seen really organize around this principle is Buildkite, which I've used and enjoyed. I'm currently using Github Actions and it's fine but Buildkite is literally sooooo good for the reasons you've mentioned.

Not to mention that the rest of the environment is missing: e.g. you probably can't push to e.g. the organization's Docker registry/Artifactory/whatever from you local dev machine (if it's even visible from your machine in the first place). And those are arguably the most interesting parts that you want to test the integration with.

I have seen both, and I consider them roughly similar.

I think GP got confused, it's not running the runners locally, it's running the CI steps locally (see the other sibling replies).

For example, running tests locally exactly the same way as in the runner - sometimes I have to open a debugger in the middle of a test to see what exactly went wrong. Our tests run in gitlab in a particular docker image, and I've been adding a "make test" that runs the same tests in the same image locally, with additional flags to have full interactivity so the debugger works if needed.

Make is one of the simplest build tools out there. Compared to something like Grunt, Webpack, etc. it’s a hammer compared to a mining drill.

The solution is to not use tools used by large corporations because they are used by large corporations. My unpopular opinion is that CI/CD is not needed in most places where it’s used. Figure out how to do your builds and deploys with the absolute fewest moving pieces even if it involves some extra steps. Then carefully consider the cost of streamlining any part of it. Buying into a large system just to do a simple thing is often times not worth it in the long run.

If you really do need CI/CD you will know because you will have a pain point. If that system is causing your developers pain, it isn’t the right fit.