What Accessibility Scanners Can and Can't Catch

Run an accessibility scanner against your site. Genuinely — do it, it takes minutes and it's free. (Ours is here, and it runs in your browser.) Then understand exactly what the result does and doesn't tell you, because that's where organizations get into trouble.

What automation is genuinely good at

Scanners excel at anything with a deterministic rule and machine-readable evidence:

• Contrast ratios between declared foreground and background colors
• Missing alt attributes, missing form labels, missing page titles
• Missing or wrong language declarations
• Some structural problems — skipped heading levels, empty links and buttons
• ARIA attributes used in invalid combinations

They're also good at scale, which is not a small thing. A scanner can crawl 40,000 pages overnight and tell you the same broken pattern appears on all of them. No human process does that.

What automation cannot evaluate

The limitation isn't a gap in the tools — it's that a large share of WCAG requires judgment about meaning:

• Is the alt text right? A scanner confirms the attribute exists. It can't tell you the photo of your mayor is described as "image1."
• Does focus order preserve meaning? It can read the DOM. It can't know what the layout communicates visually.
• Is the error message helpful? "Invalid input" passes a presence check and helps nobody.
• Does this custom component announce correctly? Roles and states can be technically valid and still describe a control that behaves nothing like what a user is told.
• Are the captions accurate? A caption track exists or it doesn't. Whether it says the right words is a human question.
• Is the video's visual content described? Entirely a judgment call.
• Can someone actually complete the task? The only question that finally matters, and no tool answers it.

The commonly cited estimate is that automated testing reliably detects on the order of a third of WCAG issues. Argue about the exact number if you like; the operational point holds. A clean scan is a starting signal, not a conformance claim.

The dangerous middle case

The worst outcome isn't a scanner missing something. It's a scanner reporting zero errors on a page that is unusable.

That happens more than you'd expect. A page built entirely from divs with click handlers can pass a scan cleanly — no missing alt text, no contrast failures, no invalid ARIA, because there's no ARIA at all. It is also completely inoperable by keyboard. The tool reports success because nothing it checks is wrong.

This is why "we ran a scan and it came back clean" should prompt more questions, not fewer.

How to use both together

The pattern that works:

1. Automate for breadth and regression. Run scanners across the whole property, and wire them into your build so new failures are caught at the pull request rather than at the next audit.
2. Test manually for depth. Keyboard-only operation, screen readers on desktop and mobile, 200% zoom, 400% reflow — on representative templates and every interactive component.
3. Test by task, not by page. Complete the actual journeys: apply, purchase, schedule, pay, contact. Page-level testing misses the failures that only appear across a flow.
4. Test with disabled users when you can. Conformance is the floor. Usability is the goal, and the two are not the same thing.

What to ask a vendor

If you're buying an accessibility audit, one question separates a real one from a repackaged scan: which assistive technologies did you test with, on which platforms, and which criteria did you evaluate manually?

A serious vendor answers specifically. If the answer is vague, you're buying a report you could have generated yourself in an afternoon.