Should you use Let's Encrypt for internal hostnames?

Running your own private CA is a great way to cause problems for yourself down the road (just ask anyone with a 5 year and 1 day old Kubernetes cluster). But I also don't want to be dependent on a 3rd party for my internal services. I want a better solution: not as annoying as a private CA, and not dependent on 3rd parties.

I want to deploy apps that use certs that don't expire. When they should be rotated, I want to do them on my own time. And I want a standard method to automatically replace them when needed, that is not dependent on some cron job firing at the correct time or everything breaks.

Cert expiration is a ticking time bomb blowing up my services just because "security best practice" says an arbitrary, hard expiration time is the best thing. Security is not more important than reliability. For a single external load balancer for a website, we deal with it. But when you have thousands of the little bastards in your backend, it's just ridiculous.

> Security is not more important than reliability.

Yes, it is. In most cases Confidentially > Integrity > Availability. Systems should fail-safe.

There are some scenarios such as medical devices where integrity or availability trump confidentiality. But most information systems should favor going offline to prevent a breach of confidentiality or data integrity.