←back to thread

No ARIA is better than bad ARIA

(www.w3.org)
125 points robin_reala | 2 comments | 03 Dec 25 08:39 UTC | HN request time: 0.382s | source
Show context
simonw ◴[09 Dec 25 10:05 UTC] No.46203241[source]▶
Something I'm desperately keen to see is AI-assisted accessibility testing.

I'm not convinced at all by most of the heuristic-driven ARIA scanning tools. I don't want to know if my app appears to have the right ARIA attributes set - I want to know if my features work for screenreader users.

What I really want is for a Claude Code style agent to be able to drive my application in an automated fashion via a screenreader and record audio for me of successful or failed attempts to achieve goals.

Think Playwright browser tests but for popular screenreaders instead.

Every now and then I check to see if this is a solved problem yet.

I think we are close. https://www.guidepup.dev/ looks extremely promising - though I think it only supports VoiceOver on macOS or NVDA on Windows, which is a shame since asynchronous coding agent tools like Codex CLI and Claude Code for web only run Linux.

What I haven't seen yet is someone closing the loop on ensuring agentic tools like Claude Code can successfully drive these mechanisms.

replies(12): >>46203277 #>>46203374 #>>46203420 #>>46203447 #>>46203583 #>>46203605 #>>46203642 #>>46204338 #>>46204455 #>>46206651 #>>46206832 #>>46208023 #
devinprater ◴[09 Dec 25 11:09 UTC] No.46203583[source]▶
There are thousands of blind people on the net. Can't you hire one of them to test for you? Please?
replies(6): >>46203654 #>>46203668 #>>46204073 #>>46204737 #>>46205153 #>>46205158 #
angiolillo ◴[09 Dec 25 14:19 UTC] No.46205158[source]▶
> There are thousands of blind people on the net. Can't you hire one of them to test for you?

Testing is a professional skill -- not all blind people are good at accessibility testing, just as not all sighted people are good at GUI testing.

My team has carved out an accessibility budget so that every couple years we can hire an accessibility consultancy (which employs a couple blind testers) for a few tens of hours of work to review one of our application workflows. Based on the issues they identify we attempt to write tests to prevent those classes of issues across the whole application suite, but our budget means that less than one percent of our UI has ever been functionally tested for accessibility.

It comes down to cost/benefit. Good testers are expensive, good accessibility testers doubly-so. And while I personally think there's a moral imperative and maybe a marketing angle, improving accessibility truthfully doesn't seem to meaningfully improve sales. But if the testing costs came down by a couple orders of magnitude it would be a complete game-changer.

replies(1): >>46207404 #
1. tracker1 ◴[09 Dec 25 17:03 UTC] No.46207404[source]▶
I think one path that could be used and wholy underrated is simply sitting with a user while they use/navigate your application. The user themselves doesn't necessarily need to be skilled tester, and you will need to have more session time than a skilled tester, but it does work and can help a lot.

Also, try using your app/site without a mouse. I've found it funny how many actual, experienced, sighted testers don't actually know the keyboard navigation for things like triggering menus, select boxes, etc. Personally, I don't think I could get used to the voice navigation myself, it's not that it doesn't work, it's just kind of noisy. Although, most sites are excessively noisy visually imo.

replies(1): >>46208143 #
2. angiolillo ◴[09 Dec 25 17:56 UTC] No.46208143[source]▶
Completely agree on both counts. We do usability testing, including with keyboard-focused advanced users.

But usability testing with blind users presents some unique challenges. A past org I worked at ran some usability studies with blind users [1] and while I was only tangentially involved in that project it seemed that subject recruitment and observations were much more complex than typical usability studies. I haven't managed to run a usability study with blind participants at my current org though we have discussed ways we could recruit blind users for studies -- our software is complex enough that we'd need someone who is both blind and a prospective user of our software.

[1] https://www.bloomberg.com/ux/2018/08/28/visually-impaired-wo...