I ditched Docker for Podman

I believe rootless containers require Linux user namespaces which have historically been the source of many vulnerabilities: https://news.ycombinator.com/item?id=43517734

I'm conflicted about whether or not it's better to run a root daemon that can launch unprivileged non-root containers or run rootless containers launched by a non-root user.

Anyone have thoughts or more definitive resources they could point to that discuss the tradeoffs?

All containers use user namespaces, but only rootless containers require unprivileged user namespaces. Unprivileged user namespaces didn't have a great security record for the first few years, but vulnerabilities are relatively rare these days.

Running podman with SELinux enforcing (the default) and with "--security-opt=no-new-privileges" combined with running applications as non-root inside their containers should further reduce the security risk. You could also disable unprivileged user namespaces inside the containers if you want, which would mean that exploiting unprivileged user namespaces would first require arbitrary code execution on the host.