←back to thread

Ask HN: How to stop an AWS bot sending 2B requests/month?

252 points lgats | 5 comments | 17 Oct 25 05:28 UTC | HN request time: 0s | source

I have been struggling with a bot– 'Mozilla/5.0 (compatible; crawler)' coming from AWS Singapore – and sending an absurd number of requests to a domain of mine, averaging over 700 requests/second for several months now. Thankfully, CloudFlare is able to handle the traffic with a simple WAF rule and 444 response to reduce the outbound traffic.

I've submitted several complaints to AWS to get this traffic to stop, their typical followup is: We have engaged with our customer, and based on this engagement have determined that the reported activity does not require further action from AWS at this time.

I've tried various 4XX responses to see if the bot will back off, I've tried 30X redirects (which it follows) to no avail.

The traffic is hitting numbers that require me to re-negotiate my contract with CloudFlare and is otherwise a nuisance when reviewing analytics/logs.

I've considered redirecting the entirety of the traffic to aws abuse report page, but at this scall, it's essentially a small DDoS network and sending it anywhere could be considered abuse in itself.

Are there others that have similar experience?

1. reisse ◴[17 Oct 25 07:40 UTC] No.45614185[source]▶
What kind of content do you serve? 700 RPS is not a big number at all, for sure not enough to qualify as a DoS. I'm not surprised AWS did not take any action.
replies(2): >>45614205 #>>45616566 #
2. marginalia_nu ◴[17 Oct 25 07:43 UTC] No.45614205[source]▶
FWIW, a HN hug of death, which fairly regularly knocks sites offline tends to peak at a few dozen RP.
replies(1): >>45614284 #
3. reisse ◴[17 Oct 25 07:58 UTC] No.45614284[source]▶
On the other hand, I've only seen complaint letters from AWS for doing tens of thousands of RPS on rate-limited endpoints for multiple days. Even then, AWS wasn't the initiator of inquiry (it was their customer being polled), and it wasn't a "cease and desist" kind of letter, it was "please explain what you're doing and prove you're not violating our ToS".
replies(1): >>45614408 #
4. hsbauauvhabzb ◴[17 Oct 25 08:22 UTC] No.45614408{3}[source]▶
Why would aws care if you’re consuming one of their customers resources when the customer is the one that pays?
5. Hizonner ◴[17 Oct 25 13:28 UTC] No.45616566[source]▶
> 700 RPS is not a big number at all, for sure not enough to qualify as a DoS.

That depends on what's serving the requests. And if you're making the requests, it is your job to know that beforehand.