An accessibility audit of oks-ui: what is still open

This is an audit of a library I maintain, so I have tried to write it the way I would want to read someone else's: what was measured, how, what it cannot tell you, and what is still wrong.

What I tested, and what I did not

• Automated checks. axe-core 4.13 with the WCAG 2.0 A and AA, WCAG 2.1 AA and best-practice rule sets, over all 59 component pages on oks-ui.com, in light and dark, in Chromium, Firefox and WebKit. That is 354 page runs.
• A keyboard-only pass. Starting at the top of each page, I pressed Tab through every control in the page body in Chromium and Firefox, about 6,800 Tab stops in each browser. For every stop I recorded the accessible name and whether the element ended up with a visible focus indicator.
• A version comparison. The library's own accessibility test has 48 examples. I rendered the same 48 with 1.1.2, 1.2.0, 1.2.2, 1.3.0, 1.3.1, 1.3.2 and 1.4.1 and ran axe on each, in light and dark.

What this does not cover: I did not test with a screen reader such as VoiceOver or NVDA, so I cannot say how any of this sounds. Automated tools only judge what they can measure, so a clean axe run is not a claim that a page is accessible. I did not test zoom or text-size changes, high-contrast mode or voice control.

Result 1: the component pages

58 of 59 component pages had no axe violations in any of the three browsers, in light or dark. The exception is the Text editor page, where text on tinted backgrounds is too faint:

• "Edit" label in the Edit / Preview toggle, light: #fe6601 on #fbe2d2, 2.37 : 1
• @mention chip, light: #fe6601 on #fbe5d7, 2.43 : 1
• Link inside the editor, light: #fe6601 on #fafafa, 2.82 : 1
• Image caption, to-do placeholder and "+ Add text" buttons, dark: #71717a on #1a1a2e, 3.52 : 1
• "Ctrl/⌘ + Enter to exit" hint, dark: #71717a on #22223a, 3.19 : 1
• "Edit" label, dark: #818cf8 on #2a2c4e, 4.48 : 1

WCAG AA asks for 4.5:1 for normal-size text. Every pair above is below it; the dark "Edit" label misses by only 0.02.

One practical note: axe can report a different answer on the same page depending on when it runs. Straight after the page load event it reported no violations on this page; after a one second pause it reported the same 4 elements in light and 8 in dark every time. If you script axe, wait for the page to settle first. The script at the end of this post does.

Result 2: the keyboard pass

Names. Every one of the roughly 6,800 Tab stops in each browser had an accessible name.

Focus indicators. Counting every distinct kind of focusable element across the pages (a button, a text field, a tab, a nav link and so on), there were 72 kinds. I took a screenshot of each one unfocused and focused with the keyboard, in light and dark, and compared the pixels. 70 of the 72 changed visibly when focused. A first version of this check read CSS properties instead of pixels and produced dozens of false alarms, because some components draw their focus ring on a different property than the one I was reading, so I discarded it. Of the other two kinds, one is a link inside the text editor demo that my script could not focus, so it is untested, and the other is the OTP field below.

The OTP field. This is the one real keyboard problem, on the OTP field page:

• Tab does not leave it. Focus in an empty field, press Tab, and focus goes straight back to the first empty box. I pressed Tab three times with nothing typed, and four times after typing three digits: every time, focus stayed in the field. It only moves on once all six digits are filled. Shift+Tab does leave. The behaviour is identical in Chromium, Firefox and WebKit.
• There is no visible focus ring. The focused box looks exactly the same as an unfocused one.

A keyboard user cannot skip an OTP field they do not want to fill in, and one who can get through it cannot see where they are. The first is at least a keyboard-flow failure, and arguably a failure of WCAG 2.1.2 (No Keyboard Trap), because Shift+Tab is the only way out.

For contrast, the segmented control is how it should look: Tab lands on the selected option, a clear orange ring appears, and the arrow keys move the selection.

What earlier releases fixed, measured

Here is the version comparison. The count is the number of elements axe flags across the 48 examples.

Nearly everything was fixed in one release, 1.2.0. Between 1.1.2 and 1.2.0 the 62 failing elements went to one. Here is every example that failed in 1.1.2:

Most were text contrast, mostly in dark mode, but not all: the Calendar had seven empty table header cells, the line Chart carried an ARIA attribute its role does not allow, and the date picker field used an ARIA attribute that is not valid on its element. Focus handling in Modal and Drawer, which axe cannot see, was also fixed in 1.2.0; I reproduced that before and after in the 1.2 release notes.

The one element still failing from 1.2.0 to 1.4.1 is the small change pill in the Stat component, in the light theme (3.34:1 when I measured it on the dashboard tutorial).

The fix a plain page cannot show

1.3.2 changed the secondary text colour and the selected Nav label. A white page does not reveal why, because the old secondary text colour passed on white (4.7:1). It failed on a lightly tinted surface, which is what many app backgrounds are:

The lesson for anyone auditing their own pages: test your real backgrounds, not just white.

Reduced motion

1.3.0 made Breadcrumbs, Pagination, SteppedForm and TextEditor turn their CSS transitions off when a reader has asked for reduced motion. I rendered them in 1.2.2 and 1.3.0 with prefers-reduced-motion: reduce emulated and measured the longest transition on each:

SteppedForm showed no transition in either version on its first step, so I could not confirm that one from the first step alone.

What is still open

Confirmed by this audit:

1. OTP field: Tab is trapped until all digits are entered, and the focus ring is invisible.
2. Text editor: the contrast failures in the table above.
3. Stat: the change pill fails contrast in the light theme.

Found earlier, while building and testing the tutorials on this site against oks-ui 1.4.1:

• Form errors in dark mode measure between 4.0 and 4.26 : 1, just under the 4.5 minimum. The same is true of the "Skipped" text in the file field.
• Table: the header checkbox does not draw its indeterminate (partly selected) state, and every row checkbox is named "Select row", so a screen reader cannot tell them apart.
• Date picker: the arrow keys do not move between days, and the calendar grid lacks the row roles a grid needs.
• File field: only the top strip of the drop area opens the file chooser when clicked, and a rejected file's message is not announced.
• Stepped form: focus is lost on the last step, Enter does not advance, and the progress bar is not exposed to assistive technology.
• Modal with `role="alertdialog"`: the message is not linked to the dialog with aria-describedby.
• Board: no on-screen preview while a card is lifted with the keyboard.
• Soft buttons: five hover and pressed combinations are below 4.5:1, the lowest at 3.85:1.

None of these is fixed yet. I have not set a release date, and I will not pretend that listing them is the same as fixing them. When one is fixed it will appear in the changelog, and this page will be updated.

Repeat the sweep on your own pages

This is the script I used for the page sweep, trimmed to one file. Install playwright and axe-core, run npx playwright install chromium, and pass it your URLs:

import { chromium } from "playwright";
import { createRequire } from "node:module";

const axePath = createRequire(import.meta.url).resolve("axe-core/axe.min.js");
const urls = process.argv.slice(2);

const browser = await chromium.launch();
for (const scheme of ["light", "dark"]) {
  const page = await (await browser.newContext({ colorScheme: scheme })).newPage();
  for (const url of urls) {
    await page.goto(url, { waitUntil: "load" });
    await page.waitForTimeout(1000); // let fonts and client rendering settle
    await page.addScriptTag({ path: axePath });
    const violations = await page.evaluate(async () =>
      (await window.axe.run(document, { runOnly: ["wcag2a", "wcag2aa", "wcag21aa"] })).violations.map((v) => ({
        rule: v.id,
        elements: v.nodes.length,
        first: v.nodes[0].target.join(" "),
      })),
    );
    console.log(scheme, url, violations.length ? violations : "no violations");
  }
}
await browser.close();

Run it as node audit.mjs http://localhost:3000/ http://localhost:3000/settings. Note that colorScheme only switches pages that follow the system theme. If your app uses a data-theme attribute, set it on the html element before the check, as I did for the version comparison.

For the keyboard, no tool replaces doing it: unplug the mouse, press Tab from the top of each page, and watch for three things. Can you reach everything? Can you always see where you are? Can you always leave?

The accessible modal post covers how to build one of those parts correctly in your own app.