I have my domain's DNS on Cloudflare, so I can use DNS verification with Let's Encrypt to get myself a proper certificate that works on all of my devices. Then I just have Cloudflare DNS set up with a bunch of CNAME records to .internal addresses.
For example, if I needed to set up a local mail server, I'd set mail.cottagecheese.download to have a CNAME record pointing to localserver.internal and then have my router resolve localserver.internal to my actual home server's IP address. So if I punch in https://mail.cottagecheese.download in my browser, the browser resolves that to localserver.internal and then my router resolves that to 10.x.x.x/32, sending me to my internal home server that greets me with a proper Let's Encrypt certificate without any need to expose my internal IP addresses.
Windows doesn't seem to like my CNAME-based setup though. Every time I try to use them, it's a diceroll if it actually works.