←back to thread

252 points lgats | 3 comments | | HN request time: 0.63s | source

I have been struggling with a bot– 'Mozilla/5.0 (compatible; crawler)' coming from AWS Singapore – and sending an absurd number of requests to a domain of mine, averaging over 700 requests/second for several months now. Thankfully, CloudFlare is able to handle the traffic with a simple WAF rule and 444 response to reduce the outbound traffic.

I've submitted several complaints to AWS to get this traffic to stop, their typical followup is: We have engaged with our customer, and based on this engagement have determined that the reported activity does not require further action from AWS at this time.

I've tried various 4XX responses to see if the bot will back off, I've tried 30X redirects (which it follows) to no avail.

The traffic is hitting numbers that require me to re-negotiate my contract with CloudFlare and is otherwise a nuisance when reviewing analytics/logs.

I've considered redirecting the entirety of the traffic to aws abuse report page, but at this scall, it's essentially a small DDoS network and sending it anywhere could be considered abuse in itself.

Are there others that have similar experience?

1. Retric ◴[] No.45623177[source]
A 100% legal solution is to sue them and name Amazon as a party in the lawsuit.

Through discovery you can get the name of the parties involved from Amazon, but Amazon is very likely to drop them as a client solving the issue.

replies(1): >>45626792 #
2. Waterluvian ◴[] No.45626792[source]
This sounds like it would probably cost tens of thousands of dollars just to get off the starting line.
replies(1): >>45626810 #
3. Retric ◴[] No.45626810[source]
Actually going through a lawsuit is expensive, “bluffing” long enough to send a nasty and credible letter can be relatively inexpensive.

Importantly it’s also getting moderately expensive for the other side which really discourages this kind of behavior. Suiting an arbitrary person you have no connection with invites a counter suit for wasting their money, but that largely goes away with such a one sided provocation.