←back to thread

XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

(xbow.com)
283 points summarity | 4 comments | 24 Jun 25 15:53 UTC | HN request time: 0.817s | source
Show context
jekwoooooe ◴[24 Jun 25 18:23 UTC] No.44369157[source]▶
They should ban this or else they will get swallowed up and companies will stop working with them. The last thing I want is a bunch of llm slop sent to me faster than a human would
replies(2): >>44369219 #>>44369231 #
danmcs ◴[24 Jun 25 18:31 UTC] No.44369231[source]▶
HackerOne was already useless years before LLMs. Vulnerability scanning was already automated.

When we put our product on there, roughly 2019, the enterprising hackers ran their scanners, submitted everything they found as the highest possible severity to attempt to maximize their payout, and moved on. We wasted time triaging all the stuff they submitted that was nonsense, got nothing valuable out of the engagement, and dropped HackerOne at the end of the contract.

You'd be much better off contracting a competent engineering security firm to inspect your codebase and infrastructure.

replies(2): >>44369339 #>>44371667 #
tptacek ◴[24 Jun 25 18:38 UTC] No.44369339[source]▶
Moreover, I don't think XBOW is likely generating the kind of slop beg bounty people generate. There's some serious work behind this.
replies(2): >>44369412 #>>44369429 #
1. tecleandor ◴[24 Jun 25 18:43 UTC] No.44369412[source]▶
Still they're sending hundreds of reports that are being refused because they are not following the rules of the bounties. So they better work on that.
replies(1): >>44369555 #
2. tptacek ◴[24 Jun 25 18:52 UTC] No.44369555[source]▶
If you thought human bounty program participants were generally following the rules, or that programs weren't swamped with slop already... at least these are actually pre-triaged vetted findings.
replies(1): >>44371313 #
3. tecleandor ◴[24 Jun 25 21:34 UTC] No.44371313[source]▶
But I was hoping the idea wasn't "as there's a lot of sloppy posts, we're going to be sloppy too let's flood them". So, use the AI for something useful and at least grep the rules properly. That'd be neat.
replies(1): >>44374333 #
4. weq ◴[25 Jun 25 06:56 UTC] No.44374333{3}[source]▶
In the first version it grepped the rules properly. By the 10th interation those rules were lost to the heavens, and replaced by a newly hallucinated set that no one noticed because everyone was now dumber.