I ditched Docker for Podman

The best CI/CD pipeline I ever used was my first freelance deployment using Django. I didn't have a clue what I was doing and had to phone a friend.

We set up a git post receive hook which built static files and restarted httpd on a git receive. Deployment was just 'git push live master'.

While I've used Docker a lot since then, that remains the single easiest deployment I've ever had.

I genuinely don't understand what docker brings to the table. I mean, I get the value prop. But it's really not that hard to set up http on vanilla Ubuntu (or God forbid, OpenBSD) and not really have issues.

Is the reproducibility of docker really worth the added overhead of managing containers, docker compose, and running daemons on your devbox 24/7?

Reproducibility? No.

Not having to regularly rebuild the whole dev environment because I need to work on one particular Python app once a quarter and its build chain reliably breaks other stuff? Priceless.

Someone wrote a PHP7 script to generate some of our daily reports a while back that nobody wants to touch. Docker happily runs the PHP7 code in the container and generates the reports on any system. its portable, and it doesnt require upkeep.

This false sense of reproducability is why I funded https://docs.stablebuild.com/ some years ago. It lets you pin stuff in dockerfiles that are normally unpinnable like OS package repos, docker hub tags and random files on the internet. So you can go back to a project a year from now and actually get the same container back again.

> I genuinely don't understand what docker brings to the table. I mean, I get the value prop. But it's really not that hard to set up http on vanilla Ubuntu (or God forbid, OpenBSD) and not really have issues.

For me, as an ex-ops, the value proposition is to be able to package a complex stack made of one or more db, several services and tools (ours and external), + describe the interface of these services with the system in a standard way (env vars + mounts points).

It massively simplify the onboarding experience, make updating the stack trivial, and also allow devs, ci and prod to run the same version of all the libraries and services.

Isn't this problem usually solved by building an actual image for your specific application, tagging that and pushing to some docker repo? At least that's how it's been at placec I've worked at that used docker. What am I missing?

> I genuinely don't understand what docker brings to the table. I mean, I get the value prop. But it's really not that hard to set up http on vanilla Ubuntu (or God forbid, OpenBSD) and not really have issues.

Sounds great if you're only running a single web server or whatever. My team builds a fairly complex system that's comprised of ~45 unique services. Those services are managed by different teams with slightly different language/library/etc needs and preferences. Before we containerized everything it was a nightmare keeping everything in sync and making sure different teams didn't step on each others dependencies. Some languages have good tooling to help here (e.g. Python virtual environments) but it's not so great if two services require a different version of Boost.

With Docker, each team is just responsible for making sure their own containers build and run. Use whatever you need to get your job done. Our containers get built in CI, so there is basically a zero percent chance I'll come in in the morning and not be able to run the latest head of develop because someone else's dev machine is slightly different from mine. And if it runs on my machine, I have very good confidence it will run on production.

Managing and running some containers is really easy though. And running daemons? Don’t we all have loads of things running all the time?

I find it easier to have the same interface for everything, where I can easily swap around ports.

Perhaps more focused on docker-based development workflows than final deployment.

what they described is a fairly common set up in damn near most enterprises

> I didn't have a clue what I was doing and had to phone a friend.

> I genuinely don't understand what docker brings to the table.

I think you invalidated your own opinion here

Use pixi or uv to maintain this specific environment and separate it from the global one

Builds typically aren’t retained forever.

Both can be true

I like how you didn't even ask for any context that would help you evaluate whether or not their chosen architecture is actually suitable for their environment before just blurting out advice that may or may not be applicable (though you would have no idea, not having enquired).

I know this pain, and Docker absolutely makes sense for this use case, but I feel like we would both agree that this is a duct tape and bubble gum solution? Though a totally justifiable one

OK, I completely agree with this.

That said, I'm not a nix guy, but to me, intuitively NixOS wins for this use case. It seems like you could either

A. Use declarative OS installs across deployments B. Put your app into a container which sometimes deploys it's own kernel and then sometimes doesn't and this gets pushed to a third party cloud registry, or you can set up your own registry, and then this container runs on a random ubuntu container or cloud hosting site where you basically don't administer or do any ops you just kind of use it as an empty vessel which exists to run your Docker container.

I get that in practice, these are basically the same, and I think that's a testament to the massive infrastructure work Docker, Inc has done. But it just doesn't make any sense to me

Sorry, sir, I didn't realize nobody should ever spend any time learning anything or, failing that, describe what happened to them during that time. I'm no neckbeard savant but I do have a dozen years of deploying web apps and also using Docker during that time, so I think I'm allowed to have an opinion. Go drink some warm milk, you will feel better.

Oh sure. 20 years ago I used VMs and that was also a duct tape solution. I'd have hoped for a proper solution by now, but a lighter hack works too

OK, this seems like an absolutely valid use case. Big enterprise microservice architecture, I get it. If you have islands of dev teams, and a dedicated CI/CD dev ops team, then this makes more sense.

But this puts you in a league with some pretty advanced deployment tools, like high level K8, Ansible, cloud orchestration work, and nobody thinks those tools are really that appropriate for the majority of devteams.

People are out here using docker for like... make install.

What do you do when you then actually need to make a change to your application (e.g. a 1-liner fix)? Edit the binary image?

Having a reproducible dev environment is great when everyone’s laptop is different and may be running different OSes, libraries, runtimes, etc.

Also docker has the network effect. If there was a good light weight tool that was better enough people would absolutely use it.

But it doesn’t exist.

In an ideal world it wouldn’t exist, but we don’t live there.

> Having a reproducible dev environment is great when everyone’s laptop is different and may be running different OSes, libraries, runtimes, etc.

Docker and other containerization solved the “it works on my machine” issue

you can actually declare containers directly in Nix. they use the same config/services/packages machinery as you'd use to declare a system config. and then you can embed them in the parent machine's config, so they all come online and talk to each other with the right endpoints and volumes and such.

or you can use the `build(Layered)Image` to declaratively build an oci image with whatever inside it. I think you can mix and match the approaches.

but yes I'm personally a big fan of Nix's solution to the "works on my machine" problem. all the reproducibility without the clunkiness of having to shell into a special dev container, particularly great for packaging custom tools or weird compilers or other finnicky things that you want to use, not serve.

QEMU used a similar CI for its website before switching to Gitlab pages:

https://gist.github.com/bonzini/1abbbdec739e77503945a3605e0e...

15 years ago i has a customer who ran a dozen different services on one machine, php, python, and others. a single dev team. upgrading was a nightmare. you upgraded one service, it broke another. we hadn't yet heard about docker, and used proxmox. but the principle is the same. this is definitely not just big enterprise.

almost. there is still an issue with selinux. i just had that case. because the client develops with selinux turned off, the docker containers don't run on my machine if i have selinux turned on.

Docker is that lightweight tool, isn’t it? It doesn’t seem that complex to me. Unfamiliar to those who haven’t used it, but not intrinsic complexity.

You can always edit the file in the container and re-upload it with a different tag. That's not best practice, but it's not exactly sorcery.

Forget docker for a second.

Suddenly you're in a team with 2-3 people and one of them likes to git push broken code and walk-off.

Okay, lets make this less about working with a jack-ass, same setup, but each 5 minutes of downtime cost you millions of dollars. One of your pushes work locally but don't work on the server.

The point of a more structed / complex CI/CD process is to eliminate failures. As the stakes become higher, and the stack becomes more complex, the need for the automation grows.

Docker is just a single part of that automation that makes other things / possible / lowers specific class of failures.

you append it to the end of the docker file so that the previous image is still valid with its cached build steps

> We set up a git post receive hook which built static files and restarted httpd on a git receive. Deployment was just 'git push live master'.

I still do that for all my personal projects! One of the advantages of docker is that you don't have to rebuild the thing on each deployment target.

> Is the reproducibility of docker really worth the added overhead of managing containers, docker compose, and running daemons on your devbox 24/7?

Yes. Everything on my box is ephemeral and can be deleted and recreated or put on another box with little-to-no thought. Infrastructure-as-code means my setup is immutable and self-documented.

It's a little more time to set up initially, but now I know exactly what is running.

I don't really understand the 24/7 comment, now that it is set up there's very very little maintenance. Sometimes an upgrade might go askew but that is rare.

Any change to it is recorded as a git commit, I don't have to worry about logging what I've done ever because it's done for me.

Changes are handled by a GitHub action, all I have to do to change what is running is commit a file, and the infra will update itself.

I don't use docker-compose, I use a low-overhead microk8s single-node cluster that I don't think about at all really, I just have changes pushed to it directly with Pulumi (in a real environment I'd use something like ArgoCD) and everything just works nicely. Ingress to services is done through Cloudflare tunnels so I don't even have to port-forward or think about NAT or anything like this.

To update my personal site, I just do a git commit/push, the it's CI/CD builds builds a container and then updates the Pulumi config in the other repo to point to the latest hash, which then kicks off an action in my infra repo to do a Pulumi apply.

Currently it runs on Ubuntu but I'm thinking of using Talos (though it's still nice to be able to just SSH to the box and mess around with files).

I'm not sure why people struggle with this, or the benefits of this approach, so much? It seems like a lot of complexity if you're inexperienced, but if you've been working with computers for a long time, it isn't particularly difficult—there are far more complicated things that computers do.

I could throw the box (old macbook) in a lake and be up and running with every service on a new box in an hour or so. Or I could run it on the cloud. Or a VPS, or metal, or whatever really, it's a completely portable setup.

I know well what you are talking about since I did something similar, but I finally moved to Docker with Kamal (except one project I still have to move). The advantage of Docker's reproducibility is to have a peace of mind when comes to rollbacks and running exact versions due to system dependencies. If anyone is curious I wrote Kamal Handbook to help people adopt Kamal which I think brings all the niceness to Docker deployment so it's not annoying.

And just keep accreting new layers indefinitely?

It's not, but at that point you're giving up on most of the things Docker was supposed to get you. What about when you need to upgrade a library dependency (but not all of them, just that one)?

The end result will be the same but I can give 3 docker commands to a new hire and they will be able to set up the stack on their MacBook or Linux or Windows system in 10 minutes.

Nix is, as far as I know, not there and we would probably need weeks of training to get the same result.

Most of the time the value of a solution is not in its technical perfection but in how many people already know it, documentation, and more important all the dumb tooling that's around it!

Imagine you have a team of devs, some using macOS, some using Debian, some using NixOS, some on Windows + WSL. Go ahead and try to make sure that everyone's development environment by simply running "git pull" and "make dev"

you miss an intermediate environment (staging, pre-prod, canary, whatever you want to call it) with selinux turned on.

Ha, I've written a lot of these Makefiles and the "make dev" command even became a personal standard that I added to each project. I don't know if I read about that, or if it just developed into that because it just makes sense. In the last few years, these commands very often started a docker container, though. I do tend to work on Windows with WSL and I most of my colleagues use macOS or Linux, so that's definitely one of the reasons why docker is just easier there.

Just for my home server, I have more than 10 containers for home assistant, vpn, library management for movies/tv/music, photos backup, password manager, and a notes server. I started without knowing what docker was, and in less than a year realized running services directly on my OS was more hassle than I wanted both with compatibility between services dependencies, networking setup for them, and configuring reboots and upgrades. I would say the reproducibility and configurability is easily worth the slight overhead and in my experience even reduced it.

i don't. the customer does. and they don't seem to care. turning selinux off works for them and they are not paying me to fix that or work around it.

> so there is basically a zero percent chance I'll come in in the morning and not be able to run the latest head of develop because someone else's dev machine is slightly different from mine.

It seems you never had to deal with timezone-dependent tests.

Build and tag internal base images on a regular cadence that individual projects then use in their FROM. You’ll have `company-debian-python:20250901` as a frozen-in-time version of all your system level dependencies, then the Dockerfile using it handles application-level dependencies with something that supports a lockfile (e.g. uv, npm). The application code itself is COPY’d into the image towards the end, such that everything before it is cached, but you’re not relying on the cache for reproducibility, since you’re starting from a frozen base image.

The base image building can be pretty easily automated, then individual projects using those base images can expect new base images on a regular basis, and test updating to the latest at their leisure without getting any surprise changes.

Docker in and of itself does not do you much good. Its strength comes from the massive amounts of generic tooling that is built around the container as the standard deployable unit.

If you want to handle all your deployments the same way, you can basically only choose between Nix and containers. Unfortunately, containers are far more popular and have more tooling.

> Is the reproducibility of docker really worth the added overhead of managing containers, docker compose, and running daemons on your devbox 24/7?

Why wouldn't it be, containers are super easy to manage, dockerd uses bugger all resources in dev (on Linux anyway) and docker compose files are the simplest setup scripts I've ever used

I like docker because it's easy and I'm lazy

I think this is accurate. It just feels like a lot of "we use Docker because everybody uses Docker. That's just the way we do it.

But if you actually add up the time we spend using docker, I'm really not sure it saves that many cycles

What are timezone-dependent tests? Sounds like a bummer

Wherefore art thou IBM

That is wild. I have been maintaining servers with many services and upgrading never broke anything, funnily enough: on Arch Linux. All the systems where an upgrade broke something were Ubuntu-based ones. So perhaps the issue was not so much about the services themselves, but the underlying Linux distribution and its presumably shitty package manager? I do not know the specifics so I cannot say, but in my case it was always the issue. Since then I do not touch any distribution that is not pacman-based, in fact, I use Arch Linux exclusively, with OpenBSD here and there.

Enterprises are frequently antipattern zoos. If you have many teams you can use the modular monolith pattern instead of microservices, that way you have the separation but not the distributed system.

i used "broken" generously. it basically means that for example for multiple php based services, we had to upgrade them all at once, which lead to a large downtime until everything was up and running again. services in containers meant that we could deal with them one at a time and dramatically reduce the downtime and complexity of the upgrade process.

Oh, I see what you mean now, okay, that makes sense.

I would use containers too, in such cases.

Would there still have been a problem if you were able to install multiple php versions side-by-side? HPC systems also have to manage multiple combinations of toolchains and environments and they typically use Modules[1] for that.

[1] https://hpc-wiki.info/hpc/Modules

You have 12 years of deployment experience and some of that using docker, would have been a more useful thing to say in your OC. I was literally just pointing out your argument was pretty weak - this context would have made it stronger.

probably not, but it wasn't just php, and also one of the goals was the ability to scale up. and so having each service in its own container meant that we could move them to different machines and add more machines as needed.

At that point you're doing most of the work yourself, and the value add from Docker is pretty small (although not zero) - most of the gains are coming from using a decent language-level dependency manager.

I once had to work with a legacy Java codebase and they hardcoded the assumption their software would run in the America/New_York timezone, except some parts of the codebase and tests used the _system_ timezone, so they would fail if run in a machine with a different timezone.

I'm not sure what the complication here is. If application code changes, or some dependency changes, you build a new docker image as needed, possibly with an updated Dockerfile as well if that's required. The Dockerfile is part of the application repo and versioned just like everything else in the repo. CICD helps build and push a new image during PRs, or tag creation, just like you would with any application package / artifact. Frequent building and pushing of docker images can over time start taking up space of course but you can take care of that by maybe cleaning out old images from time to time if you can determine they're no longer needed.

Docker re-uses layers as needed and can detect when a new layer needs to be added. It's not like images grow in size without bound each time something is changed in the Dockerfile.

> God forbid, OpenBSD

What exactly is your problem with OpenBSD? Shaming it completely out of context is kinda mean - they're the upstream for OpenSSH and LibreSSL.

Yeah most enterprise software barely works and is an absolute maintenance nightmare because they're sprawling distrivuted systems.

Ask yourself: how does an enterprise with 1000 engineers manage to push a feature out 1000x slower than two dudes in a garage? Well, processes, but also architecture.

Distributed systems slow down your development velocity by many orders of magnitude, because they create extremely fragile systems and maintenance becomes extremely high risk.

We're all just so used to the fragility and risk we might think it's normal. But no, it's really not, it's just bad. Don't do that.

Much like parallel programming, distributed systems have a very small window of requirement.

Really less than 1% of systems need to be distributed. Are you Google? No? Then you probably don't need it.

The rest is just for fun. Or, well, pain. Usually pain.

I like how you didn't even enquire as to what size organisation they worked in order to determine if it might actually be applicable in their case.