Most active commenters

    ←back to thread

    Ask HN: How to stop an AWS bot sending 2B requests/month?

    252 points lgats | 12 comments | 17 Oct 25 05:28 UTC | HN request time: 0.525s | source | bottom

    I have been struggling with a bot– 'Mozilla/5.0 (compatible; crawler)' coming from AWS Singapore – and sending an absurd number of requests to a domain of mine, averaging over 700 requests/second for several months now. Thankfully, CloudFlare is able to handle the traffic with a simple WAF rule and 444 response to reduce the outbound traffic.

    I've submitted several complaints to AWS to get this traffic to stop, their typical followup is: We have engaged with our customer, and based on this engagement have determined that the reported activity does not require further action from AWS at this time.

    I've tried various 4XX responses to see if the bot will back off, I've tried 30X redirects (which it follows) to no avail.

    The traffic is hitting numbers that require me to re-negotiate my contract with CloudFlare and is otherwise a nuisance when reviewing analytics/logs.

    I've considered redirecting the entirety of the traffic to aws abuse report page, but at this scall, it's essentially a small DDoS network and sending it anywhere could be considered abuse in itself.

    Are there others that have similar experience?

    1. AdamJacobMuller ◴[17 Oct 25 16:38 UTC] No.45618776[source]▶
    > I've tried 30X redirects (which it follows)

    301 response to a selection of very large files hosted by companies you don't like.

    When their AWS instances start downloading 70000 windows ISOs in parallel, they might notice.

    Hard to do with cloudflare but you can also tar pit them. Accept the request and send a response, one character at a time (make sure you uncork and flush buffers/etc), with a 30 second delay between characters.

    700 requests/second with say 10Kb headers/response. Sure is a shame your server is so slow.

    replies(5): >>45619101 #>>45621437 #>>45622490 #>>45623571 #>>45628839 #
    2. notatoad ◴[17 Oct 25 17:05 UTC] No.45619101[source]▶
    >301 response to a selection of very large files hosted by companies you don't like.

    i suggest amazon

    replies(2): >>45621192 #>>45623837 #
    3. lgats ◴[17 Oct 25 19:46 UTC] No.45621192[source]▶
    unfortunately, it seems AWS even has firewalls that will quickly start failing these requests after a few thousand, then they're back up to their high-concurrency rate
    4. gruez ◴[17 Oct 25 20:09 UTC] No.45621437[source]▶
    >When their AWS instances start downloading 70000 windows ISOs in parallel, they might notice.

    Inbound traffic is free for AWS

    replies(1): >>45622803 #
    5. gitgud ◴[17 Oct 25 21:46 UTC] No.45622490[source]▶
    > Accept the request and send a response, one character at a time

    Sounds like the opposite of the [1] Slow Loris DDOS attack. Instead of attacking with slow connections, you’re defending with slow connections

    [1] https://www.cloudflare.com/en-au/learning/ddos/ddos-attack-t...

    replies(1): >>45624096 #
    6. jacquesm ◴[17 Oct 25 22:18 UTC] No.45622803[source]▶
    It's free, but it's not infinite.
    7. tremon ◴[18 Oct 25 00:05 UTC] No.45623571[source]▶
    As an alternative: 301 redirect to an official .sg government site, let local law enforcement deal with it.
    replies(1): >>45626940 #
    8. knowitnone3 ◴[18 Oct 25 00:49 UTC] No.45623837[source]▶
    Microsoft
    9. tliltocatl ◴[18 Oct 25 01:35 UTC] No.45624096[source]▶
    That's why it is actually sometimes called inverse slow loris.
    replies(1): >>45624618 #
    10. amy_petrik ◴[18 Oct 25 03:18 UTC] No.45624618{3}[source]▶
    it's called the slow sirol in my circles
    11. integralid ◴[18 Oct 25 12:50 UTC] No.45626940[source]▶
    Don't actually do this, unless you fancy meeting AWS lawyers in court and love explaining intricate details of HTTP to judges.
    12. more_corn ◴[18 Oct 25 17:13 UTC] No.45628839[source]▶
    ^ I love you