By: Trenton Moss

December 15th, 2005

Reader Rating: 8.5

An automated accessibility tool is a piece of software that can test a Web page, or even an entire Website, for accessibility. Automated accessibility tools are useful because they can save you a huge amount of time. Don't want to check images for alt text on each and every page on your Website? Run the site through an automated tester and it'll do all the checking for you!

Automated accessibility testing tools have been around for a long time and have historically been a useful way of checking the accessibility of Websites. Bobby, one of the first and most well-known automated accessibility testing tools, is now almost 10 years old, and although it's no longer freely available, plenty of other free tools such as WebXact [1] and Wave [2] now exist.

But are these tools a little too good to be true? I think so. Can you really test a Website for accessibility so easily? Unfortunately, the answer is a resounding no. Numerous problems are associated with using automated tools alone to test a site's accessibility.

Automated Tools Interpret Accessibility Guidelines Literally

Being a piece of software, an automated accessibility testing tool doesn't have much in the way of common sense. It will interpret each and every accessibility guideline literally, without giving any thought to what other things are happening on the page.

The definition of the word "guideline", according to Dictionary.com, is "a rule or principle that provides guidance to appropriate behaviour". A guideline simply offers guidance to what constitutes best practice: it shouldn't be applied as-is, without regard to other factors.

For example, one of the W3C accessibility guidelines [3] states that a summary should be provided for every table [4]. This summary doesn't appear on the screen, but is read aloud to screen reader users before the table content is read. Table summaries are useful because they tell screen reader users what to expect in the table. Great! ...But imagine you've placed a heading directly before the table, which also describes what the table is about. In this instance, this summary is essentially redundant as it will likely repeat the heading. Yet automated accessibility tools would identify this as an issue, regardless.

Automated Tools can't Check Content Accessibility

The way that content is structured -- both on the page and across the site -- is a massive part of a site's accessibility. A Website may be perfectly coded, and conform to the highest coding standards but, if its content is poorly structured, the site will prove difficult to impossible to use for some visitors with special needs.

Automated accessibility testing tools are unable to check for a number of important content accessibility considerations, including:

• The "front-loading" of content, so that each paragraph begins with the conclusion (checkpoint 13.8 [5])
• Ensuring content has been broken into manageable chunks preceded by descriptive sub-headings (checkpoint 12.3 [6])
• Using lists wherever appropriate (checkpoint 3.6 [7])
• Ensuring that plain and simple language is used (checkpoint 14.1 [8])

Automated Tools Can't Check Many Coding Issues

The vast number of accessibility guidelines tend to be related to the way a site is coded. However, automated accessibility testing tools are unable to test for adherence to many of these guidelines. Automated tools can't check that:

• Text is real text, and isn't embedded within images (checkpoint 3.1 [9])
• The site functions without the use of JavaScript or Flash (checkpoint 6.3 [10])
• Equivalent text links are provided where server-side image maps are used (checkpoint 1.2 [11])
• The structure within the HTML reflects the page's visual appearance (e.g. headings are labeled as headings within the HTML code) (checkpoints 5.1, 3.5, 3.6, 3.7, 5.4 [12])

If we rely on automated tools alone, we'll be unable to check that these important guidelines have been met.

Automated Tools use Outdated Guidelines in Many Cases

Automated accessibility testing tools generally use the W3C accessibility guidelines, which are now over five years old. A number of these guidelines are outdated and no longer apply. In fact, some of them are now thought to hinder accessibility rather than help, so it's best to ignore these guidelines.

For example, an automated accessibility testing tool will probably insist that form items contain default place-holding text (checkpoint 10.4 [13]). It may also insist that links be separated by non-link text (checkpoint 10.5 [14]). Neither of these guidelines is relevant any more; in fact, their implementation could make accessibility worse rather than better. (It must be noted that the W3C states that these two guidelines are only relevant as long as user agents continue to require them; nonetheless, most automated tools still fail a Web page for violating these guidelines.)

Automated Tools don't Check most Guidelines Properly

Automated accessibility tools can check a number of guidelines, and tell you when a guideline isn't being met. However, in some instances in which the tool claims that a guideline is being fulfilled, this may, in fact, be incorrect.

For example, if all images contain alt text, the software will report a pass for this guideline. But what if the alt text isn't descriptive of its image? What if alt text is crammed full of nonsensical keywords for search engines? How can an automated accessibility tool possibly know this?

Warnings Generated by Automated Tools may be Misinterpreted

The reports generated by automated accessibility tools provide warnings, as well as errors. These warnings are basically guidelines for which the automated tool can't check, but which may be errors. Often they're not; in fact, often they're not even relevant. However, some people who read a report that contains such notifications may try to get rid of the warning messages by making the appropriate changes to their site. By doing so, they may be implementing guidelines that needn't be implemented, and in the process, inadvertently reducing the Website's accessibility.

Don't Fall into the Automated Tool Trap!

Automated accessibility testing tools can be useful -- they can save you a large amount of time in performing some very basic accessibility checks. However, they must be used with caution, and cannot be used as a stand-alone guide for accessibility checking. Indeed, expert accessibility knowledge should always be applied in evaluating a site's accessibility, perhaps in conjunction with the fantastic Web accessibility toolbar for Internet Explorer [15] or the equally excellent Web Developer Toolbar for Firefox [16], both of which can help speed up manual checks dramatically.

[1] http://webxact.watchfire.com
[2] http://wave.webaim.org
[3] http://www.w3.org/TR/WCAG10/full-checklist.html
[4] http://www.w3.org/TR/
[5] http://www.w3.org/TR/
[6] http://www.w3.org/TR/
[7] http://www.w3.org/TR/
[8] http://www.w3.org/TR/
[9] http://www.w3.org/TR/
[10] http://www.w3.org/TR/
[11] http://www.w3.org/TR/
[12] http://www.w3.org/TR/WCAG10/full-checklist.html
[13] http://www.w3.org/TR/
[14] http://www.w3.org/TR/
[15] http://www.nils.org.au/ais/web/resources/toolbar
[16] http://chrispederick.com/work/firefox/webdeveloper/