←back to thread

Ask HN: How did the internet discover my subdomain?

287 points govideo | 1 comments | 06 Mar 25 22:34 UTC | HN request time: 0.607s | source

I have a domain that is not live. As expected, loading the domain returns: Error 1016.

However...I have a subdomain with a not obvious name, like: userfileupload.sampledomain.com

This subdomain IS LIVE but has NOT been publicized/posted anywhere. It's a custom URL for authenticated users to upload media with presigned url to my Cloudflare r2 bucket.

I am using CloudFlare for my DNS.

How did the internet find my subdomain? Some sample user agents are: "Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7; en-us) AppleWebKit/534.20.8 (KHTML, like Gecko) Version/5.1 Safari/534.20.8", "Mozilla/5.0 (Linux; Android 9; Redmi Note 5 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.89 Mobile Safari/537.36",

The bots are GET requests which are failing, as designed, but I'm wondering how the bots even knew the subdomain existed?!

Show context
yatralalala ◴[07 Mar 25 12:49 UTC] No.43289743[source]▶
Hi, our company does this basically "as-a-service".

The options how to find it are basically limitless. Best source is probably Certificate Transparency project as others suggested. But it does not end there, some other things that we do are things like internet crawl, domain bruteforcing on wildcard dns, dangling vhosts identification, default certs on servers (connect to IP on 443 and get default cert) and many others.

Security by obscurity does not work. You can not rely on "people won't find it". Once it's online, everyone can find it. No matter how you hide it.

replies(13): >>43289843 #>>43290143 #>>43290420 #>>43290596 #>>43290783 #>>43292505 #>>43292547 #>>43292687 #>>43293087 #>>43303762 #>>43309048 #>>43317788 #>>43341607 #
sl1ckback[dead post] ◴[07 Mar 25 14:48 UTC] No.43290596[source]▶
[flagged]
MyOutfitIsVague ◴[07 Mar 25 15:37 UTC] No.43290999[source]▶
That's not what that phrase means. That's not even what the word "obscure" means. Obscurity is trying to not draw attention to something, or keep it hidden (as in "nobody knows that it's there", not "you know that it's there but can't access it"). Encryption doesn't obscure data unless you're stretching the definition of the word beyond its useful purpose.
replies(3): >>43291017 #>>43291091 #>>43291455 #
elliotbnvl ◴[07 Mar 25 15:38 UTC] No.43291017[source]▶
verb: keep from being seen; conceal.

In what way is what he’s describing not obscurity?

replies(6): >>43291146 #>>43291167 #>>43291173 #>>43291201 #>>43291220 #>>43291341 #
1. dghlsakjg ◴[07 Mar 25 15:54 UTC] No.43291173[source]▶
Yes that is what the word obscure means.

But the phrase “security through obscurity” is an industry term that refers to keeping things secure purely by not letting people know they exist.

In contrast with encryption, where I can tell you exactly where the encrypted data is, but you can’t access it.

Security through obscurity is hiding a bicycle in a bush and hoping no one notices it, encryption is more like locking it to a bike rack with a very good lock.

replies(1): >>43291256 #