Your Accessibility Score Is Not a Compliance Strategy
An automated accessibility score measures what a scanner can detect, which is a minority of the WCAG success criteria. A site can score 95 and still be impossible to use with a keyboard. Scores are useful for watching direction over time. They are not evidence of conformance, and they will not hold up.
I sell a scanner. I want to be very clear about what it does before I tell you what it doesn't.
What do automated accessibility scanners actually test?
Scanners are good at the things that have a right answer in the markup. Does this image have an alt attribute. Do these two colors meet 4.5:1. Does this form control have a programmatic label. Is there an <html lang>. Does the heading order skip a level.
Those are decidable. A computer can look at the page, compare it to a rule, and be correct.
How much a scanner sees depends on where it looks. One that parses static HTML never runs your JavaScript, so every ARIA state your framework sets at runtime and every control your app renders after load is invisible to it. A scanner that drives a real browser sees the page your users actually get. That difference is most of why I built eiSEO the way I did, and it is a real difference. It is also not the difference between a third of WCAG and all of it.
Because then there is everything else. Whether the alt text describes the image or just says "image". Whether the focus order matches the visual order in a way a person can follow. Whether an error message tells you how to fix the problem. Whether the reading order of a three-column layout makes sense out loud. Whether a custom dropdown behaves like a dropdown when you press Escape.
Those require judgment. No scanner has judgment. Mine included, and its own product page says so: what it cannot judge without a person, like reading order, it says so rather than guessing.
I build one of these, so take this as an insider's estimate rather than a citation: the honest coverage number is about a third of the WCAG success criteria, and I would not defend a tighter figure than that. Even a third is generous, because a criterion can be partially testable. Contrast is the clean example. A scanner catches text on a solid background and misses the same text sitting on a photo.
| What a scanner catches reliably | What it cannot judge |
|---|---|
| Missing alt attributes | Whether the alt text is accurate or useful |
| Contrast failures on solid backgrounds | Contrast of text over images, video, or gradients |
| Missing form labels | Whether the label makes sense to a human |
| Skipped heading levels | Whether the headings describe the content |
| Controls with no keyboard path at all | Whether focus can get back out once it gets in |
| Positive tabindex and scrambled DOM order | Whether the focus order makes sense to a person following it |
| Custom widgets with no keyboard handler bound | Whether the widget behaves like the thing it imitates |
That right-hand column is where the lawsuits live.
One note on versions, since it comes up. The DOJ Title II rule sets WCAG 2.1 Level AA as the standard. Good scanners, including mine, test against 2.2 AA. Since 2.1 is a subset of 2.2, testing to the newer version clears the regulatory bar and then some. It still only clears the part a machine can see.
Can a site score 95 and still lock people out?
Yes, and I've watched it happen.
Picture a public-facing site with a clean scan. Green across the board. Alt text present on everything. Contrast passing. Headings in order. The team is proud of it, and they should be, because that work is real.
Then someone tabs into the appointment scheduler. It's a custom modal. Tab moves focus into it fine. Tab does not move focus back out. Escape does nothing. The only way out is to reload the page.
That is a keyboard trap. WCAG 2.1.2, Level A, the lowest bar there is.
Notice what the scanner could and could not have done there. It can tell you a control has no keyboard path, or that someone sprinkled positive tabindex values around, or that a focusable element is hidden from assistive technology. Those are states it can read. A trap is not a state. It is something you only discover by pressing Tab thirty times and noticing you stopped going anywhere. No tool presses Tab thirty times and forms an opinion about being stuck.
The score said 95. For the person using a keyboard, the site was zero. There is no partial credit on "I could not complete the task."
What are the three manual tests anyone can run in fifteen minutes?
You do not need training or a license for these. You need your own site, your own hands, and fifteen minutes. Do them on your top task, not your homepage. The homepage is where nobody converts.
Keyboard only, five minutes. Put the mouse behind the monitor. Load your most important page. Use Tab, Shift+Tab, Enter, Space, and the arrow keys to complete one real task start to finish: find the thing, fill the form, submit it. Watch for three failures. Can you always see where focus is. Does focus ever get stuck. Does focus ever jump somewhere that makes no sense, like the top of the page after opening a menu.
200% zoom, five minutes. In your browser, press Ctrl and plus (Cmd and plus on a Mac) until you hit 200%. Do not resize the window, do not use a phone. Now read the page. Does content overlap. Does anything get cut off with no way to scroll to it. Did the navigation disappear entirely instead of collapsing. Can you still finish the task. WCAG 1.4.4 and 1.4.10 both live here, and text-heavy government and clinic sites fail this constantly.
Screen reader on the single top task, five minutes. Turn on the one already installed: VoiceOver on a Mac (Cmd + F5), Narrator on Windows (Ctrl + Win + Enter), or NVDA if you want the tool most blind users actually run. Close your eyes for the first thirty seconds. Listen to the page load. Then navigate by headings and try to complete the same task. You are not auditing. You are asking one question: could I have done this without looking.
Write down what broke. That list is worth more than any score you have ever generated, including mine.
I put the whole thing on one sheet, with the specific WCAG criteria each test covers and a table to record findings in: The Fifteen Minute Accessibility Test. Free, no email required, and the PDF is tagged and screen-reader tested, because it would be embarrassing if it weren't.
How should you use a score, then?
As a trend line, per site, over time. Not as a certificate.
A score is a decent answer to "are we getting better or worse this quarter." Scanning fifty sites monthly and watching which ones drift after a redesign or a new editor joins is genuinely useful, and it is the reason automated scanning exists at all. You cannot manually test fifty sites every month. Nobody can.
What a score cannot do is answer "are we conformant." Conformance is a claim about all of WCAG, not the part a machine can see.
So: use the scan to find the cheap wins and to catch regressions. Use manual testing to find the things that actually stop people. Use both, and never confuse the first for the second.
What do auditors and plaintiffs actually look at?
Not your dashboard.
They look at whether a person with a disability could complete a task on your site. That's it. The evidence is usually a video of someone failing to do something ordinary: register for a class, pay a bill, book an appointment, download the agenda for a public meeting.
Nobody has ever been saved in a demand letter by a screenshot of a green score. What does help is documentation of an ongoing process: what you test, how often, what you found, what you fixed, and what you have not fixed yet with a date next to it. Good faith is a real thing, and it is made of records, not numbers.
There are 196 days until April 26, 2027.
If your entire accessibility program is a monthly scan, you have a monitoring system and not a compliance strategy. The good news is that the fifteen minutes above will tell you more about where you stand than the last six months of scans did.
Run eiSEO if you want the part a machine can actually check handled and tracked without thinking about it. That's what it's for, and I'd rather tell you its limits than have you find them in a deposition. Then go put the mouse behind the monitor.
Key points
- Automated accessibility scanners detect a minority of WCAG success criteria, so a high score can coexist with a site that is unusable by keyboard.
- Running the scan in a real browser rather than parsing static HTML catches considerably more, including runtime ARIA states and controls that render after load. It still does not catch what requires judgment.
- The failures that matter most, like whether focus can get back out of a modal and whether alt text is actually useful, are the ones no tool can judge.
- Three manual tests take fifteen minutes total: keyboard only, 200% browser zoom, and a screen reader on your single top task.
- Use scores as per-site trend lines to catch regressions, not as evidence of conformance.
- Auditors and plaintiffs look at whether a person could complete a task, and at your documented process, not at your dashboard.