←back to thread

XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

(xbow.com)
283 points summarity | 5 comments | 24 Jun 25 15:53 UTC | HN request time: 0.87s | source
Show context
ryandrake ◴[24 Jun 25 18:09 UTC] No.44369008[source]▶
Receiving hundreds of AI generated bug reports would be so demoralizing and probably turn me off from maintaining an open source project forever. I think developers are going to eventually need tools to filter out slop. If you didn’t take the time to write it, why should I take the time to read it?
replies(7): >>44369097 #>>44369153 #>>44369155 #>>44369386 #>>44369772 #>>44369954 #>>44370907 #
tptacek ◴[24 Jun 25 19:07 UTC] No.44369772[source]▶
These aren't like Github Issues reports; they're bug bounty programs, specifically stood up to soak up incoming reports from anonymous strangers looking to make money on their submissions, with the premise being that enough of those reports will drive specific security goals (the scope of each program is, for smart vendors, tailored to engineering goals they have internally) to make it worthwhile.
replies(1): >>44370208 #
1. ryandrake ◴[24 Jun 25 19:41 UTC] No.44370208[source]▶
Got it! The financial incentive will probably turn out to be a double edged sword. Maybe in the pre-AI age, it’s By Design to drive those goals, but I bet the ability to automate submissions will inevitably alter the rules of these programs.

I think within the next 5 years or so, we are going to see a societal pattern repeating: any program that rewards human ingenuity and input will become industrialized by AI to the point where it becomes a cottage industry of companies flooding every program with 99% AI submissions. What used to be lone wolves or small groups of humans working on bounties will become truckloads of AI generated “stuff” trying to maximize revenue.

replies(2): >>44371154 #>>44371611 #
2. dcminter ◴[24 Jun 25 21:16 UTC] No.44371154[source]▶
I'm wary of a lot of AI stuff, but here:

> What used to be lone wolves or small groups of humans working on bounties will become truckloads of AI generated “stuff” trying to maximize revenue.

You're objecting to the wrong thing. The purpose of a bug bounty programme is not to provide a cottage industry for security artisans - it's to flush out security vulnerabilities.

There are reasonable objections to AI automation in this space, but this is not one of them.

3. t0mas88 ◴[24 Jun 25 22:12 UTC] No.44371611[source]▶
Might be fixable by adding a $ 100 submission fee that is returned when you're proving working exploit code. Would make the Curl team a lot of money.
replies(1): >>44378062 #
4. billy99k ◴[25 Jun 25 14:54 UTC] No.44378062[source]▶
I've been on Hackerone for almost 8 years and I think the problem with this is that too many companies won't pay for legitimate bugs, even when you have a working exploit.

I had one critical bug take 3 years to get a pay out. I had a full walkthrough with videos and report. The company kept stalling and at one point told me that because they completely had the app remade, they weren't going to pay me anything.

Hackerone doesn't really protect the researcher either. I was told multiple times that there was 'nothing they could do'.

I eventually got paid, but this is pretty normal behavior with regards to bug bounty. Too many companies use it for free security work.

replies(1): >>44379665 #
5. tptacek ◴[25 Jun 25 17:13 UTC] No.44379665{3}[source]▶
I do think HackerOne is problematic, in that it pushes companies that don't really understand bug bounties to stand up bounty programs without a clear reason. If you're doing a serious bounty, your incentive is to pay out. But a lot of companies do these bounties because they just think they're supposed to.

Most companies should not do bug bounties.