←back to thread

Ask HN: How did the internet discover my subdomain?

287 points govideo | 5 comments | 06 Mar 25 22:34 UTC | HN request time: 1.936s | source

I have a domain that is not live. As expected, loading the domain returns: Error 1016.

However...I have a subdomain with a not obvious name, like: userfileupload.sampledomain.com

This subdomain IS LIVE but has NOT been publicized/posted anywhere. It's a custom URL for authenticated users to upload media with presigned url to my Cloudflare r2 bucket.

I am using CloudFlare for my DNS.

How did the internet find my subdomain? Some sample user agents are: "Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7; en-us) AppleWebKit/534.20.8 (KHTML, like Gecko) Version/5.1 Safari/534.20.8", "Mozilla/5.0 (Linux; Android 9; Redmi Note 5 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.89 Mobile Safari/537.36",

The bots are GET requests which are failing, as designed, but I'm wondering how the bots even knew the subdomain existed?!

Show context
yatralalala ◴[07 Mar 25 12:49 UTC] No.43289743[source]▶
Hi, our company does this basically "as-a-service".

The options how to find it are basically limitless. Best source is probably Certificate Transparency project as others suggested. But it does not end there, some other things that we do are things like internet crawl, domain bruteforcing on wildcard dns, dangling vhosts identification, default certs on servers (connect to IP on 443 and get default cert) and many others.

Security by obscurity does not work. You can not rely on "people won't find it". Once it's online, everyone can find it. No matter how you hide it.

replies(13): >>43289843 #>>43290143 #>>43290420 #>>43290596 #>>43290783 #>>43292505 #>>43292547 #>>43292687 #>>43293087 #>>43303762 #>>43309048 #>>43317788 #>>43341607 #
TZubiri ◴[07 Mar 25 13:06 UTC] No.43289843[source]▶
"Security by obscurity does not work"

This is one of those false voyeur OS internet tennets designed to get people to publish their stuff.

Obscurity is a fine strategy, if you don't post your source that's good. If you post your source, that's a risk.

The fact that you can't rely on that security measure is just a basic security tennet that applies to everything: don't rely on a single security measure, use redundant barriers.

Truth is we don't know how the subdomain got leaked. Subdomains can be passwords and a well crafted subdomain should not leak, if it leaks there is a reason.

replies(16): >>43290226 #>>43290237 #>>43290330 #>>43290608 #>>43290616 #>>43290675 #>>43290677 #>>43290740 #>>43290760 #>>43291317 #>>43291775 #>>43291815 #>>43292414 #>>43292523 #>>43292777 #>>43295244 #
1. zevlag ◴[07 Mar 25 14:03 UTC] No.43290226[source]▶
> Subdomains can be passwords and a well crafted subdomain should not leak,

I disagree. A subdomain is not secret in any way. There are many ways in which it is transmitted unencrypted. A couple:

- DNS resolution, multiple resolvers and authoritative servers - TLS SNI - HTTP Host Header

There are many middle boxes that could perform safety checks on behalf of the client, and drop it into a list to be rescanned.

- Virus Scanners - Firewalls - Proxies

replies(2): >>43291470 #>>43294259 #
2. dharmab ◴[07 Mar 25 16:23 UTC] No.43291470[source]▶
I once worked for a company which was using a subdomain of an internal development domain to do some completely internal security research on our own products. The entire domain got flagged in Safe Browsing despite never being exposed to the outside world. We think Chrome's telemetry flagged it, and since it was technically routable as a public IP (all public traffic on that IP was blackholed), Chrome thought it was a public website.
replies(1): >>43291813 #
3. mkl95 ◴[07 Mar 25 16:57 UTC] No.43291813[source]▶
I saw a similar thing happen with a QA team's domains. Google flagged them as malicious and the company never managed to get them unflagged.
replies(1): >>43292060 #
4. dharmab ◴[07 Mar 25 17:24 UTC] No.43292060{3}[source]▶
Our lawyers knew their lawyers so there was a friendly chat and we got added to an internal whitelist within Google.
5. TZubiri ◴[07 Mar 25 20:32 UTC] No.43294259[source]▶
>It's not encrypted in transit

Agree.

But who said that all passwords or shiboleths should all be encrypted in transit?

It can serve as a canary for someone snooping your traffic. Even if you encrypt it, you don't want people snooping.

To date of my subdomains that I never publish, I haven't had anyone attempting to connect with them.

It's one of those redundant measures.

And it's also one of those risks that you take, you can maximize security by staying at home all day, but going out to take the trash is a calculated risk that you must take or risk overfocusing on security.

It's similar to port knocking. If you are encrypting it, it's counterproductive, it's a low effort finishing touch, like a nice knot.