←back to thread

XBOW, an autonomous penetration tester, has reached the top spot on HackerOne

(xbow.com)
283 points summarity | 8 comments | 24 Jun 25 15:53 UTC | HN request time: 0s | source | bottom
Show context
tecleandor ◴[24 Jun 25 18:41 UTC] No.44369385[source]▶
First:

> To bridge that gap, we started dogfooding XBOW in public and private bug bounty programs hosted on HackerOne. We treated it like any external researcher would: no shortcuts, no internal knowledge—just XBOW, running on its own.

Is it dogfooding if you're not doing it to yourself? I'd considerit dogfooding only if they were flooding themselves in AI generated bug reports, not to other people. They're not the ones reviewing them.

Also, honest question: what does "best" means here? The one that has sent the most reports?

replies(2): >>44369528 #>>44372234 #
1. jamessinghal ◴[24 Jun 25 18:51 UTC] No.44369528[source]▶
Their success rates on HackerOne seem widely varying.

  22/24 (Valid / Closed) for Walt Disney

  3/43 (Valid / Closed) for AT&T
replies(2): >>44369569 #>>44370666 #
2. thaumasiotes ◴[24 Jun 25 18:53 UTC] No.44369569[source]▶
> Their success rate on HackerOne seems widely varying.

Some of that is likely down to company policies; Snapchat's policy, for example, is that nothing is ever marked invalid.

replies(1): >>44369633 #
3. jamessinghal ◴[24 Jun 25 18:57 UTC] No.44369633[source]▶
Yes, I'm sure anyone with more HackerOne experience can give specifics on the companies' policies. For now, those are the most objective measures of quality we have on the reports.
replies(1): >>44369795 #
4. moyix ◴[24 Jun 25 19:09 UTC] No.44369795{3}[source]▶
This is discussed in the post – many came down to individual programs' policies e.g. not accepting the vulnerability if it was in a 3rd party product they used (but still hosted by them), duplicates (another researcher reported the same vuln at the same time; not really any way to avoid this), or not accepting some classes of vuln like cache poisoning.
5. pclmulqdq ◴[24 Jun 25 20:27 UTC] No.44370666[source]▶
Walt Disney doesn't pay bug bounties. AT&T's bounties go up to $5k, which is decent but still not much. It's possible that the market for bugs is efficient.
replies(1): >>44371772 #
6. monster_truck ◴[24 Jun 25 22:36 UTC] No.44371772[source]▶
Walt Disney's program covers substantially more surface area, there's 6? publicly traded companies listed there. In addition to covering far fewer domains & apps, AT&T's conditions and exclusions disqualify a lot more.

The market for bounties is a circus, breadcrumbs for free work from people trying to 'make it'. It can safely be analogized to the classic trope of those wanting to work in games getting paid fractional market rates for absurd amounts of QA effort. The number of CVSS vulns with a score above 8 that have floated across the front page of HN in the past year without anyone getting paid tells you that much.

replies(1): >>44376144 #
7. ackbar03 ◴[25 Jun 25 11:43 UTC] No.44376144{3}[source]▶
> The market for bounties is a circus, breadcrumbs for free work from people trying to 'make it'. > The number of CVSS vulns with a score above 8 that have floated across the front page of HN in the past year without anyone getting paid tells you that much.

You make it sound like there's a ton of people going around who can just dig up CVSS vulns above 8 and is making me all confused. Is that really happening? I have a single bounty on H1 just to show I could do it, and that still took ages and was a shitty bug.

replies(1): >>44379047 #
8. monster_truck ◴[25 Jun 25 16:22 UTC] No.44379047{4}[source]▶
The weighted average is 7.6. Finding them doesn't necessarily take much effort if you know what to look for.