←back to thread

Be Aware of the Makefile Effect

(blog.yossarian.net)
431 points thunderbong | 10 comments | 11 Jan 25 04:13 UTC | HN request time: 0s | source | bottom
Show context
mcdeltat ◴[11 Jan 25 06:44 UTC] No.42663837[source]▶
At my work I've noticed another contributing factor: tools/systems that devs need to interact with at some point, but otherwise provide little perceived value to learn day-to-day.

Example is build system and CI configuration. We absolutely need these but devs don't think they should be expected to deal with them day to day. CI is perceived as a system that should be "set and forget", like yeah we need it but really I have to learn all this just to build the app? Devs expect it to "just work" and if there are complexities then another team (AKA my role) deals with that. As a result, any time devs interact with the system, there's a high motivation to copy from the last working setup and move on with their day to the "real" work.

The best solution I see is meet the devs halfway. Provide them with tooling that is appropriate simple/complex for the task, provide documentation, minimise belief in "magic". Tools like Make kinda fail here because they are too complex and black-box-like.

replies(6): >>42663956 #>>42663968 #>>42664489 #>>42666168 #>>42666871 #>>42669136 #
nicoburns ◴[11 Jan 25 07:12 UTC] No.42663968[source]▶
For me the big problems with CI setups tend to be:

- They're often slow

- They're often proprietary

- They're often dealing with secrets which limits who can work on them

- You generally can't run them locally

So the feedback cycle for working on them is incredibly long. And working on them is therefore a massive pain.

replies(3): >>42664170 #>>42664775 #>>42666180 #
1. tempodox ◴[11 Jan 25 14:41 UTC] No.42666180[source]▶
> You generally can't run them locally

GitLab CI gives you local runners. You can completely self-host CI.

replies(2): >>42666715 #>>42666940 #
2. Joker_vD ◴[11 Jan 25 15:59 UTC] No.42666715[source]▶
Well, yes, but aren't those runners have different configuration than the runners that are actually deployed and used by your company's CI/CD?
replies(1): >>42666899 #
3. tempodox ◴[11 Jan 25 16:25 UTC] No.42666899[source]▶
Wat? Our company's CI is GitLab CI, self-hosted. What other runners would we need?
replies(1): >>42667013 #
4. Plasmoid ◴[11 Jan 25 16:31 UTC] No.42666940[source]▶
It's not self hosting. It's can I run the build from my local command line and get the same results.
replies(1): >>42666971 #
5. tempodox ◴[11 Jan 25 16:37 UTC] No.42666971[source]▶
How is running those runners on our own hosts (locally) not self-hosting?
replies(1): >>42667532 #
6. Joker_vD ◴[11 Jan 25 16:43 UTC] No.42667013{3}[source]▶
The local ones, mentioned in the original comment that we've been discussing in this thread? The local runner, executing on the developer's machine.
replies(1): >>42667474 #
7. tempodox ◴[11 Jan 25 17:42 UTC] No.42667474{4}[source]▶
Of course it makes no difference whether that runner runs on our local CI host or on my dev machine.
replies(1): >>42668621 #
8. mdaniel ◴[11 Jan 25 17:48 UTC] No.42667532{3}[source]▶
Spoken like someone who has not tried what you are describing. There are two moving parts to your response: a locally hosted runner awaits jobs from GitLab itself, which doesn't help running _locally_, and the other part is that -- back when it existed! -- trying $(gitlab-runner exec) was not a full fledged implementation of the GitLab CI concepts, making it the uncanny valley of "run something locally."

However, as of v16 there is no more exec https://gitlab.com/gitlab-org/gitlab/-/issues/385235 which I guess is good and bad. Good in that it not longer sets improper expectations that it could have plausibly done anything, and bad in that now it joins GitHub Actions[1] in not having any _local_ test strategy aside from "boot up gitlab/gitlab-ce && echo good luck"

1: yes, I'm acutely aware of the 3(?) implementations/forks of nektos/act that claim to do GHA but, again, readme is not software and I can tell you with the utmost certainty they do not do as advertised

replies(1): >>42668357 #
9. Joker_vD ◴[11 Jan 25 19:36 UTC] No.42668357{4}[source]▶
Not to mention that the rest of the environment is missing: e.g. you probably can't push to e.g. the organization's Docker registry/Artifactory/whatever from you local dev machine (if it's even visible from your machine in the first place). And those are arguably the most interesting parts that you want to test the integration with.
10. Izkata ◴[11 Jan 25 20:18 UTC] No.42668621{5}[source]▶
I think GP got confused, it's not running the runners locally, it's running the CI steps locally (see the other sibling replies).

For example, running tests locally exactly the same way as in the runner - sometimes I have to open a debugger in the middle of a test to see what exactly went wrong. Our tests run in gitlab in a particular docker image, and I've been adding a "make test" that runs the same tests in the same image locally, with additional flags to have full interactivity so the debugger works if needed.