Accessibility Testing Standards

As required by the Illinois Information Technology Accessibility Act (IITAA) and the Americans with Disabilities Act (ADA), testing is performed to confirm conformance with:

• Web Content Accessibility Guidelines (WCAG) 2.1 Level AA (https://www.w3.org/TR/WCAG21)
• Accessible Rich Internet Applications (ARIA) 1.2
  (https://www.w3.org/TR/wai-aria-1.2)

Accessibility testing is performed with the following tools/techniques:

• Accessibility Insights for Web (https://accessibilityinsights.io/downloads)
• Colour Contrast Analyser (https://www.tpgi.com/color-contrast-checker)
• High Contrast & Zoom (https://doit.illinois.gov/initiatives/accessibility/testing/visual) 
• Keyboard Commands (https://doit.illinois.gov/initiatives/accessibility/testing/keyboard)

Additional testing may be performed by trained testers using the following assistive technologies:

• NVDA screen reader (https://www.nvaccess.org)
• JAWS screen reader (https://www.freedomscientific.com/products/software/jaws)
• ZoomText screen magnifier (https://www.freedomscientific.com/products/software/zoomtext)
• Dragon speech recognition (https://www.nuance.com/dragon/business-solutions/dragon-professional) 

If functional impact ratings are provided, the following scale is used:

• Critical – Will prevent some users from completing essential task(s).
• High – Will require some users to take extra steps to complete essential task(s). 
• Med – May be confusing to some users, but task(s) can be completed without extra steps. 
• Low – Violates accessibility standards, but not likely to affect users.
  

Accessibility testing includes the following steps:

1. Automated Tests
2. Visual Tests
3. Keyboard Tests
4. Assistive Technology Tests
   

Automated accessibility testing is useful but has limitations:

• Automated testing tools are able to test fewer than 60% of accessibility criteria, so additional manual testing is necessary.
• Current automated testing tools are susceptible to false positives, so findings must be confirmed through manual testing.

Because of these limitations, the State of Illinois uses automated testing to:

1. Provide an initial overview of potential accessibility issues, and
2. Identify elements that need extended manual testing.

For automated testing, Illinois primarily uses Microsoft’s Accessibility Insights for Web, as it (and other tools based on the axe-core engine) has a low incidence of false positives and is able to test within shadow DOMs.

To test with Accessibility Insights: 

1. Browse to the screen to be tested. 
2. If there are elements of the screen that are not initially displayed, such as collapsed or pop-up sections, display as many of them as possible. (If there are elements that cannot be displayed simultaneously, re-run the test in each state.)
   
3. Open Accessibility Insights and select "FastPass" or "Automated checks: List view and filtering".
4. Export or make note of the failed instances for confirmation with manual tests. 
5. If there are findings regarding ARIA, or any other issues that are unclear, refer them to the DoIT Office of Information Accessibility (DoIT.Accessibility@Illinois.gov) for confirmation.

Note: Do not just copy and paste failure details from Accessibility Insights. Findings must be confirmed and clearly documented in terms that developers will understand.

Visual testing is performed using a combination of a Window High Contrast and browser Zoom:

1. Set your display resolution or browser window to 1280 px wide.
2. Press left Alt + left Shift + PrtSc (Print Screen). If prompted to turn on High Contrast, click Yes. Windows will activate a theme with a black background, white text, and blue or yellow links. (If High Contrast does not activate, check Windows Control Panel > Ease of Access Center > Make the computer easier to see > Turn on or off High Contrast.)
3. In the browser, open the Settings menu (Alt + F) and set Zoom to 200%.
4. Browse to the screen to be tested. 
5. If there are elements of the screen that are not initially displayed, such as collapsed sections or pop-up dialogs, display as many of them as possible.
6. Visually review all elements of the screen except logos, decorative (meaningless) images, or images of text that are duplicative of text provided elsewhere on the screen. Check for:
   • Text that did not increase in size by 200%. (WCAG 1.4.4)
   • Text that did not take on high-contrast colors. (WCAG 1.4.5)
   • Text that was cut off, overlapped, or otherwise became unreadable. (WCAG 1.4.4)
   • Information that was conveyed by color that is now now distinguishable by any means. (WCAG 1.4.1)
7. Press left Alt + left Shift + PrtSc to turn off High Contrast.
8. Follow the instructions in the Color Contrast Guide to measure the contrast of any color combinations except in logos, decorative images, duplicative images, or disabled elements. Check for and document:
   • Regular-sized text with a contrast ratio less than 4.5:1. (WCAG 1.4.3)
   • Large text (24px or 19px bold, or larger) with a contrast ratio less than 3:1. (WCAG 1.4.3)
   • User interface components (e.g., links, buttons, etc.) or graphics (e.g., icons, charts, etc.) with a contrast ratio less than 3:1 (WCAG 1.4.11)
9. If any of the issues listed above are found, report a failure. In the report, clearly identify the element involved, which check it failed, and any steps required to reproduce the failure. 
10. For color contrast failures, clearly identify the element(s) and include the hexadecimal color codes of the foreground and background colors and the computed color contrast ratio, e.g., Links are light blue (#6699DD) on white (#FFFFFF) with a contrast ratio of 2.9:1. The text should be darkened to meet the minimum of 4.5:1 (e.g. #4477BB).

Keyboard testing is performed using standard keyboard commands only – a mouse must not be used. Keyboard testing may be completed by following existing test scripts substituting keyboard commands for mouse actions. Otherwise, keyboard testing should be performed by doing the following on each page/screen:

1. Browse to the screen to be tested.
2. Explore the screen with the mouse to identify and determine the function of all interactive elements.
3. Click in the browser address bar and press enter to reload the page; then set the mouse aside.
4. If necessary, press the Tab key several times to move focus past any browser toolbars and into the page.
5. Once focus is in the page, use the following keyboard commands to move to and operate all interactive elements: 
   • Tab - Move to the next interactive element. 
     
   • Shift + Tab - Move backwards to the previous interactive element (when needed). 
   • Right or Left Arrow - Move horizontally through items in a menu bar, tab list, or grid. 
   • Up or Down Arrow - Move vertically through items in a dropdown, list box, radio button group, menu, or grid. 
   • Alt + Down Arrow - Open a dropdown list or calendar popup. 
   • Spacebar - Check a checkbox or click a button. 
     
   • Enter - Click a link.
   • Alt + Left Arrow - After clicking a link, go back to the previous page.  
   • Other - For any other controls, look up the Keyboard Interactions in the ARIA Authoring Practices Guide (APG) (https://www.w3.org/WAI/ARIA/apg/patterns)
6. At each step, check for and document any: 
   • Interactive elements that do not receive keyboard focus. (WCAG 2.1.1)
   • Elements that do not show a visual indicator (e.g., outline) when focused. (WCAG 2.4.7)
   • Elements that do not recieve focus in a logical order (e.g., left to right, top to bottom). (WCAG 2.4.3)
   • Elements that cause unexpected changes (e.g., reload the page) when they receive focus. (WCAG 3.2.1)
   • Elements than cannot be operated using standard keyboard commands. (WCAG 2.1.1)
     
   • Elements that cause unexpected changes when values are changed. (WCAG 3.2.2)
   • Elements that trap focus (i.e., prevent focus from moving to the next element). (WCAG 2.1.2)
7. If any of the issues listed above are found, report a failure. In the report, clearly identify the element involved, which check it failed, and any steps required to reproduce the failure. 

Assistive technology testing should be performed only by testers who have been trained to use assistive technology tools. Do not test with assistive technology unless you are completely confident in your ability to use it as it would be used by someone with a disability.

Assistive technology testing may be performed on a representative sample of screens/pages. The representative sample should include screens required for the essential functions of the application and any design patterns or interface component not present on the essential screens.

Screen reader testing should be performed using the browser and screen reader most likely to be used by the intended audience(s) of the system, i.e., Edge with JAWS for internal applications and Chrome with NVDA for public applications.

Use factory default settings and do not make any setting changes other than to speech rate (and, for NVDA, “Highlight Navigator Object”). NEVER use a mouse, and rely ONLY on what you can hear (i.e., do not look at the screen to fill in gaps in what you're hearing).

Perform screen reader testing in two passes: 

1. Read through all content using Down Arrow, and 
2. Read through and operate interactive elements using Tab and other keyboard commands.

Follow these steps:

1. Browse to the screen to be tested.
2. Explore the screen to identify all content and functions of interactive elements.
3. Start the screen reader.
4. Click in the browser address bar and press Enter to reload the page; then set the mouse aside.
5. Press Esc to make sure you’re not in Forms/Focus Mode.
6. Press Ctrl + Home to read the page title (JAWS) or first line (NVDA).
7. Press Down Arrow to read through all content.
8. If you don’t hear something clearly, press Up Arrow and then Down Arrow to re-read it.
9. If Forms/Focus Mode turns on (with a pop or chime), press Esc (once or twice) and then continue pressing Down Arrow
10. For each line, check:
    • Text – any text displayed on the screen should be read (WCAG 1.1.1)
    • Headings – any text that is large and/or bold at the beginning of a section should be indicated as a “heading” of the appropriate level, e.g., “heading level 1” for the main heading, “heading level 2” for section headings, “heading level 3” for sub-section headings, etc.) (WCAG 1.3.1)
    • Lists – any text that has bullets, numbers, or otherwise looks like a list, should be indicated as a “list of x items” (WCAG 1.3.1)
    • Images – meaningful images should be indicated as a “graphic” followed by a concise statement of what the image communicates (not a description of what it looks like); If an image does not communicate anything (i.e., is decorative), should be not be read. (1.1.1)
    • Tables – anything that looks like a table of data should be indicated as a “table of x rows and y columns” or a “grid” (WCAG 1.3.1). If you encounter a table, test it by doing the following:
      • Press Ctrl + Alt + Right Arrow  – cell content and column number should be read. Repeat until end of row/edge of table.
      • Press Down Arrow – content of first cell in next row should be read. Repeat until in the first cell in the body of the table.
      • Press Ctrl + Alt + Right Arrow – column header(s), column number, and content of cell should be read. Repeat until end of row/edge of table.
      • Press Ctrl + Alt + Down Arrow – row header (optionally), row number, and content of cell should be read. Repeat until bottom of column/edge of table.
    • Order – all content should be read in a logical order, usually left to right, top to bottom within any sections or columns of the page (WCAG 1.3.2)
    • Note: wait until pass 2 to test links & form fields.

After reading the entire page, go back to the top check all interactive elements using the Tab key:

1. Press Ctrl + Home to read the title and/or first line.
2. Press the Tab key to move to and read the next interactive element.
3. If you don’t hear something clearly, press Insert + Tab to re-read it.
4. For each interactive element, check: 
   • Links – links should be indicated as “link” (or “visited link” or “same page link”) with text that clearly and uniquely indicates the destination or purpose of the link (WCAG 2.4.3)
   • Form Fields – name, role value, and state should be indicated. Use standard keyboard commands to ensure that all elements of compound widgets are clearly identified and operable (WCAG 4.1.2)
     • Textbox – [label] Edit [value?] [multiline?] [readonly?]
     • Dropdown – [label] Combobox [value] [collapsed/expanded]
     • Listbox - [label] List [item x of y]
     • Checkbox – [label] Checkbox [checked/not checked]
     • Radio button – [group label] Group [label] Radio Button [checked/not checked] [x of y]
     • Button – [label] Button
     • Other interactive elements – look up WAI-ARIA Roles, States, and Properties in the ARIA Authoring Practices Guide

If screen magnifier testing is performed, it should be completed using ZoomText with full-screen magnification set to 4 x, color enhancement active, and speech disabled.

Confirm that all information is viewable and all interactive elements are operable using standard keyboard commands.

If speech recognition testing is performed, it should be completed using Dragon Professional and/or Windows Speech Recognition. Voice training should be completed and recognition accuracy confirmed before testing.

Confirm that all interactive elements are operable using voice commands. Preference should be given to commands targeting specific elements, such as “click first name.” Mouse movement commands, such as Mouse Grid, should be used only if specific commands do not work. 

Revised 1/20/2025