←back to thread

Building a CI/CD Pipeline Runner from Scratch in Python

(muhammadraza.me)
55 points mr_o47 | 2 comments | 09 Nov 25 19:26 UTC | HN request time: 0.404s | source
Show context
systemerror ◴[12 Nov 25 17:52 UTC] No.45903306[source]▶
Why does air-gapped environment require rolling your own CI/CD solution? Plenty of examples of air-gapped Jenkins and/or Argo Workflows. Was this just an educational exercise?
replies(3): >>45904073 #>>45906749 #>>45906809 #
verdverm ◴[12 Nov 25 21:10 UTC] No.45906749[source]▶
Jenkins sucks but is insanely reliable

Argo Workflows does not live up to what they advertise, it is so much more complex to setup and then build workflows for. Helm + Argo is pain (both use the same template delimiters...)

replies(1): >>45907793 #
bigstrat2003 ◴[12 Nov 25 22:31 UTC] No.45907793[source]▶
Jenkins, like many tools with extreme flexibility, sucks as much as you make it suck. You can pretty easily turn Jenkins into a monstrosity that makes everyone afraid to ever try to update it. On the other hand, you can also have a pretty solid setup that is easy to work on. The trouble is that the tool itself doesn't guide you much to the good path, so unless you've seen a pleasant Jenkins instance before you're likely to have a worse time than necessary.
replies(1): >>45907969 #
IshKebab ◴[12 Nov 25 22:47 UTC] No.45907969[source]▶
Are you sure, because last time I used Jenkins it actively sucked. The interface was a total mess and it doesn't surface results in any useful way.
replies(2): >>45908393 #>>45908424 #
1. blackjack_ ◴[12 Nov 25 23:31 UTC] No.45908424[source]▶
What particular issues do you have with it? My company uses it at scale (dozens of different instances, hundreds of workers, thousands of pipelines) to support thousands of applications and we are reasonably happy with it. DSL is incredibly helpful at scale. IAC is incredibly helpful at scale. It requires a good amount of upkeep, but all things underpinning large amounts of infrastructure require a good amount of upkeep.
replies(1): >>45908837 #
2. verdverm ◴[13 Nov 25 00:22 UTC] No.45908837[source]▶
We've minimized our usage of the DSL, there is no way for devs to debug it without pushing commits, and it means you have to implement much of your CI logic twice (once for local dev, once for ci system).

IMO, ci should be running the same commands humans would run (or could if it is production settings). Thus our Jenkins pipelines became a bunch of DSL boilerplate wrapped around make commands. The other nice thing about this is that it prepares you for easier migrations to a new ci system