←back to thread

253 points akyuu | 2 comments | | HN request time: 1.002s | source
1. erickhill ◴[] No.45946415[source]
I very much relate to the author's sour mood and frustration. I also host a small hobby forum and have experienced the same attacks constantly, and it has gotten especially bad the last couple of years with the rise of AI.

In the early days I put Google Analytics on the site so I could observe traffic trends. Then, we were all forced to start adding certificates to our sites to keep them "safe".

While I think we're all doomed to continue that annual practice or get blocked by browsers, I have often considered removing Google Analytics. Ever since their redesign it is essentially unusable for me now. What benefit does it bring if I can't understand the product anymore?

Last year, in a fit of desperation, I added Cloudflare. This has a brute force "under attack" mode that seems to stop all bots from accessing the site. It puts up a silly "hang on a second, are you human" page before the site loads, but it does seem to work. It is great UX? No, but at least the site isn't getting hammered by various locations in Asia. Cloudflare also let me block entire countries, although that seems to be easily fooled.

I also don't think a lot of the bots/AI crawlers honor the rules set in the robots.txt. It's all an honor system anyway, and they are completely lacking in it.

There need to be some hard and fast rules put in place, somehow, to stop the madness.

replies(1): >>45947194 #
2. timpera ◴[] No.45947194[source]
Cloudflare does work, but it often destroys the experience for legitimate users. On the website I manage, non-technical users were often getting stuck on the Cloudflare captcha, so I ended up removing it.

Then there's also the issue with dependence to US-based services, but that may not be an issue for you.