←back to thread

Ask HN: How did the internet discover my subdomain?

287 points govideo | 4 comments | 06 Mar 25 22:34 UTC | HN request time: 0s | source

I have a domain that is not live. As expected, loading the domain returns: Error 1016.

However...I have a subdomain with a not obvious name, like: userfileupload.sampledomain.com

This subdomain IS LIVE but has NOT been publicized/posted anywhere. It's a custom URL for authenticated users to upload media with presigned url to my Cloudflare r2 bucket.

I am using CloudFlare for my DNS.

How did the internet find my subdomain? Some sample user agents are: "Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7; en-us) AppleWebKit/534.20.8 (KHTML, like Gecko) Version/5.1 Safari/534.20.8", "Mozilla/5.0 (Linux; Android 9; Redmi Note 5 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.89 Mobile Safari/537.36",

The bots are GET requests which are failing, as designed, but I'm wondering how the bots even knew the subdomain existed?!

Show context
vince14 ◴[07 Mar 25 00:06 UTC] No.43286319[source]▶
I'm having the same issue.

https://securitytrails.com/ also had my "secret" staging subdomain.

I made a catch-all certificate, so the subdomain didn't show up in CT logs.

It's still a secret to me how my subdomain ended up in their database.

replies(3): >>43286374 #>>43286394 #>>43286463 #
1. johnklos ◴[07 Mar 25 00:30 UTC] No.43286463[source]▶
Serious question: Do you really think that Cloudflare is trying to keep these kinds of thing private? If so, I'd suggest that's not a reasonable expectation.
replies(1): >>43288267 #
2. fc417fc802 ◴[07 Mar 25 08:29 UTC] No.43288267[source]▶
Related question (not rhetorical). If you do DNS for subdomains yourself (and just use Cloudflare to point dns.example.com at your box) will the subdomain queries leak and show up in aggregate datasets? What I'm asking is if query recursion is always handled locally or if any of the reasonably common software stacks resolve it remotely.
replies(2): >>43290504 #>>43326647 #
3. immibis ◴[07 Mar 25 14:37 UTC] No.43290504[source]▶
As well as assuming Cloudflare sells DNS lists, it's probably safe to assume the operators of public resolvers like 8.8.8.8, 9.9.9.9 and 1.1.1.1 (that is Google, Quad9 and Cloudflare again) are looking at their logs and either selling them or using them internally.
4. johnklos ◴[10 Mar 25 22:07 UTC] No.43326647[source]▶
If you just use Cloudflare as a registrar, then they can't see what resolution happens on your servers.

If you delegate a subdomain through Cloudflare to your own DNS servers, from what I remember from the animal book, the recursive server should ask Cloudflare for the address of the machine to which the delegation has been made (yours), and while any further resolutions would be answered by your machine, Cloudflare would at very least know of every query to that subdomain.

If you delegate a subdomain and have subdomains under that subdomain, then Cloudflare would only see resolutions to that subdomain and not to the sub-subdomains.

In other words, for most things, they'd have full insight.