←back to thread

XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

(xbow.com)
283 points summarity | 1 comments | 24 Jun 25 15:53 UTC | HN request time: 0s | source
Show context
mkagenius ◴[24 Jun 25 18:19 UTC] No.44369114[source]▶
> XBOW submitted nearly 1,060 vulnerabilities.

Yikes, explains why my manually submitted single vulnerability is taking weeks to triage.

replies(2): >>44369323 #>>44374572 #
tptacek ◴[24 Jun 25 18:37 UTC] No.44369323[source]▶
The XBOW people are not randos.
replies(1): >>44369375 #
lcnPylGDnU4H9OF ◴[24 Jun 25 18:40 UTC] No.44369375[source]▶
That's not their point, I think. They're just saying that those nearly 1060 vulnerabilities are being processed so theirs is being ignored (hence "triage").
replies(1): >>44369411 #
tptacek ◴[24 Jun 25 18:43 UTC] No.44369411[source]▶
If that's all they're saying then there isn't much to do with the sentiment; if you're legit-finding #1061 after legit-findings #1-#1060, that's just life in the NFL. I took instead the meaning that the findings ahead of them were less than legit.
replies(3): >>44369640 #>>44369966 #>>44370374 #
croes ◴[24 Jun 25 18:58 UTC] No.44369640[source]▶
Whether it is legit-finding is precisely what needs to be checked, but you’re at spot 1061.

>130 resolved

>303 were classified as Triaged

>33 reports marked as new

>125 remain pending

>208 were marked as duplicates

>209 as informative

>36 not applicable

20% bind a lot of resources if you have a high input on submissions and the numbers will rise

replies(1): >>44369723 #
tptacek ◴[24 Jun 25 19:03 UTC] No.44369723[source]▶
I think some context I probably don't share with the rest of this thread is that the average quality of a Hacker One submission is incredibly low. Like however bad you think the median bounty submission is, it's worse; think "people threatening to take you to court for not paying them for their report that they can 'XSS' you with the Chrome developer console".
replies(3): >>44370182 #>>44370892 #>>44371926 #
croes ◴[24 Jun 25 19:39 UTC] No.44370182{5}[source]▶
We‘ll get this low quality submissions with AI too.

The problem is that the people who know how to use AI properly will slower and more careful in their submissions.

Many others won’t, so we‘ll get lots of noise hiding the real issues. AI makes it easy to produce many bad results in short time.

replies(1): >>44370554 #
tptacek ◴[24 Jun 25 20:14 UTC] No.44370554{6}[source]▶
Everyone already agrees with that; the interesting argument here is that it also makes it easy to produce many good results in short time.
replies(1): >>44371865 #
croes ◴[24 Jun 25 22:48 UTC] No.44371865{7}[source]▶
But the good ones don’t have the same output rate because they are checked by humans before submission.

They are faster than the purely manual ones but can’t beat the AI created bad ones neither in speed nor numbers.

It’s like the IT security version of the Gish gallop.

replies(1): >>44372127 #
1. tptacek ◴[24 Jun 25 23:21 UTC] No.44372127{8}[source]▶
Then you're refuting the premise of the article, and you should be more specific in your critique, because right now all you're saying is "this can't work".