←back to thread

Ask HN: How did the internet discover my subdomain?

287 points govideo | 10 comments | 06 Mar 25 22:34 UTC | HN request time: 1.796s | source | bottom

I have a domain that is not live. As expected, loading the domain returns: Error 1016.

However...I have a subdomain with a not obvious name, like: userfileupload.sampledomain.com

This subdomain IS LIVE but has NOT been publicized/posted anywhere. It's a custom URL for authenticated users to upload media with presigned url to my Cloudflare r2 bucket.

I am using CloudFlare for my DNS.

How did the internet find my subdomain? Some sample user agents are: "Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7; en-us) AppleWebKit/534.20.8 (KHTML, like Gecko) Version/5.1 Safari/534.20.8", "Mozilla/5.0 (Linux; Android 9; Redmi Note 5 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.89 Mobile Safari/537.36",

The bots are GET requests which are failing, as designed, but I'm wondering how the bots even knew the subdomain existed?!

Show context
yatralalala ◴[07 Mar 25 12:49 UTC] No.43289743[source]▶
Hi, our company does this basically "as-a-service".

The options how to find it are basically limitless. Best source is probably Certificate Transparency project as others suggested. But it does not end there, some other things that we do are things like internet crawl, domain bruteforcing on wildcard dns, dangling vhosts identification, default certs on servers (connect to IP on 443 and get default cert) and many others.

Security by obscurity does not work. You can not rely on "people won't find it". Once it's online, everyone can find it. No matter how you hide it.

replies(13): >>43289843 #>>43290143 #>>43290420 #>>43290596 #>>43290783 #>>43292505 #>>43292547 #>>43292687 #>>43293087 #>>43303762 #>>43309048 #>>43317788 #>>43341607 #
1. ◴[07 Mar 25 14:06 UTC] No.43290254[source]▶
2. remlov ◴[07 Mar 25 14:08 UTC] No.43290264[source]▶
If you look at the company they founded it's a service to protect yourself. Not to willy-nilly go out into the open web to find hidden subdomains.
3. tmerc ◴[07 Mar 25 14:12 UTC] No.43290304[source]▶
Why would enumerating a wildcard dns through brute force be something that evokes pride or shame?
replies(1): >>43290707 #
4. ivell ◴[07 Mar 25 14:29 UTC] No.43290449[source]▶
Irrespective of whether they are proud of what they are doing, I found the post helpful and educational. Let's not prevent people from sharing their knowledge as it might help us to protect ourselves. A consequence of such line of questioning would be that in future they would be hesitant to share their knowledge to avoid being judged.
5. yatralalala ◴[07 Mar 25 15:01 UTC] No.43290707[source]▶
I sadly did not see the comment above, but I'd like to just add, that this bruteforce and sniffing methods are target only against our paying customers.

We built global reverse-DNS dataset solely from cert transparency logs. Our active scanning/bruteforcing runs only for assets owned by our customers.

replies(1): >>43292379 #
6. 6stringmerc ◴[07 Mar 25 17:57 UTC] No.43292379{3}[source]▶
…as long as your tools are only in your hands to be used, correct? Once a tool is created and used on a machine with access to the greater internet, doesn’t your logic hold that its security is compromised inherently? Not saying you have been infiltrated, or a rogue employee has cleverly exported a copy or the methodology to duplicate it off-site, but I’m not saying that hasn’t happened either.
replies(2): >>43292575 #>>43295191 #
7. lxgr ◴[07 Mar 25 18:09 UTC] No.43292512[source]▶
Given that bad actors can also do this, I'd say that publicly advertising the fact and thereby drawing attention to misconceptions about security is a net good thing.
8. cryptonector ◴[07 Mar 25 18:14 UTC] No.43292575{4}[source]▶
It's not that hard to write this code. It's not a nuclear weapon.
replies(1): >>43292855 #
9. lkt ◴[07 Mar 25 21:57 UTC] No.43295191{4}[source]▶
You can find a dozen projects on Github that do this, it's not sensitive information that needs protecting
10. BLKNSLVR ◴[07 Mar 25 23:27 UTC] No.43295943[source]▶
I assumed they do it for customers who pay them to determine their security profile.