How to Read a VPAT

Check out our webinar, How to Read a VPAT.

What is a VPAT

A VPAT is an overview of the accessibility compliance of a product. It is often owned and assessed by the product’s vendor – though that’s not necessarily a good idea. It is required by the Section 508 Refresh and almost all Federal solicitations.

What a VPAT is not

A VPAT is not an accessibility audit report of a product – although, it is necessary to conduct an accessibility audit of the product to write an accurate VPAT. Without an accessibility audit, a VPAT is just a guess about the product’s accessibility compliance.

Interestingly, VPATs are not just for government. A lot of organizations are now asking for them, as they are really the only way to get an accurate sense of the accessibility compliance of a product. It’s also not just for web products – a VPAT can also be provided for desktop applications (for example, Word), hardware (for example, a smartphone) and equipment (for example, a photocopier).

VPATs only cover the accessibility compliance of the output of a product. Therefore, if you are looking at a product that has some authoring components (such as a Content Management System or video player) you will need to determine if the product allows for authoring accessible content (outputting valid code in the case of a CMS, or having a feature for audio descriptions in the case of a video player).

Do not assume that a VPAT is accurate, especially in the case where a VPAT has been written by the product’s vendor. However, there are cases where an external agency has written the VPAT, and it is still inaccurate.

Sections of a VPAT

A VPAT has the following sections:

• Title: “[Company Name] Accessibility Conformance Report”
• Name of product and version
• Product description
• Report date
• Contact information
• Notes
• Evaluation Methods Used
• Report information
• Terms
• Tables for each standard or guideline

How to tell if a VPAT is reasonable in ten minutes or less

The following are some red flags – indications that a VPAT may not be accurate:

• Cells without content in the table section
  Each cell should have content. If a product meets a requirement, there must be a description on how this is achieved. If a product does not meet a requirement or only partially meets a requirement then this must also be described in detail.
• Terminology such as “Passes” and “Fails”
  The standard terminology for a VPAT is “Supports,” “Partially Supports” and “Does Not Support.” If a VPAT does not use this terminology then the author is not an accessibility specialist. As an audit of the product must be conducted prior to the development of a VPAT, if the author is not an accessibility specialist then this does not bode well.
• A lot of “NA”
  Many VPATs say “NA” instead of “Does Not Support.” Only when a product does not have a feature can the specification “NA” be used. For example, if a VPAT for a video player specified “NA” for the Captions requirement, then that would be a red flag.
• All “Supports”
  It is very unlikely that the product is completely accessible, especially if the VPAT contains other red flags.
• One VPAT for multiple products
  There should be one VPAT per product. Having a single VPAT for multiple products makes tracking requirements that are not supported very difficult.
• An inaccurate or unclear description of the product
  This often means that the VPAT author does not know the product. Therefore, any accessibility assessment of the product will be flawed.
• The Notes section specifying that the VPAT does not cover essential features
  An example of this would be a VPAT for a CMS that specifically excluded the WYSIWG editor, or a VPAT for a website that does not cover the registration process.
• Dated more than 12 months ago
  VPATs should be updated when a product is updated. A VPAT that is more than 12 months old will not be representative of the current product, unless, of course, the product hasn’t been updated in 12 months – in which case, accessibility is probably the least of your problems!
• Does not contain details of testing undertaken (the “Evaluation methods used” section)
  The Section 508 Refresh added this section to the VPAT, and it is an excellent indication of whether the VPAT author has expertise in accessibility, as well as whether an accessibility audit has been undertaken.
• Only automated testing listed in “Evaluation Methods”
  Automated testing tools can only test approximately 30% of all errors (yes, even AccessibilityOz’ own OzART can only test this percentage!); if an audit consists only of automated testing then the accessibility compliance cannot be accurately judged.
• Missing or inaccurate Contact information
  This is an indication that accessibility is not a priority with the vendor. They may have employed someone to assist in the accessibility of the product, but if that person has left and has not been replaced then the accessibility compliance of the product is likely to deteriorate.
• “Not evaluated” in the Level A or Level AA section
  This means that this product wasn’t tested. Often, VPAT authors specify this instead of having to put “Does Not Support”. Wherever there is “Not evaluated” in the Level A or AA section, read this as “Does Not Support.”

Contacting the vendor

If you have identified some red flags then it is essential that you contact the vendor. Example questions are:

• “Why is this dated 14 months ago? Has the product not been updated in 14 months?”
• “Why did you say this section is NA when the product has this feature?”
• “What kind of testing did you do to ensure compliance?”
• “Why did you decide to fill out this VPAT internally?”
• “Why does the description omit essential features of the product?”
• “Why did you decide to test with only one screen reader?”

It is very important to assess how the vendor responds. If they have a good answer – one that sounds reasonable and is well thought out, then they probably do care about accessibility and may just have forgotten to update their VPAT.

What does a good VPAT look like?

A good VPAT has the following characteristics:

• Has explanations for all criteria
• Uses conventional wording such as “Supports,” “Partially Supports,” “Does Not Support” and “Not applicable”
• Stands up to questioning
• Has been written by someone other than the product vendor
• Has a clear description of the product, including associated features
• Refers to existing accessibility testing tools in the testing undertaken

How do you assess the accuracy of a VPAT?

1.     Read through the VPAT

Does the VPAT reference known accessibility issues with the product?

If you don’t know the product, Google it! The best search terms are “XYZ accessibility,” where XYZ is the common name of the product. If that doesn’t provide much information, have a look through the WebAIM mailing list archives, or send an email to the group. Someone before you has looked at the accessibility of this product and has something to say. There’s no need to reinvent the wheel!

Does the VPAT accurately describe the product?

Read through the description –does it accurately describe the product and its essential features? Are parts of the product excluded?

Does the VPAT include reasonable Evaluation Methods?

Is there manual testing as well as automated testing? Did they test with assistive technologies? If they used a specific testing tool, ask for the output of that tool.

Does the VPAT use correct terminology?

Does the VPAT use “Supports,” “Partially Supports,” “Does Not Support” and “Not applicable”?

Are the tables completed accurately?

Are there “Not Applicables” for features the product definitely has? Is each requirement given a Comment? When reading a Comment for a requirement the product “Partially Supports,” is it clear what parts of the product meet the requirement and which parts don’t? Can the product be used, or does this mean that there is essential functionality that is inaccessible?

2.     Test the VPAT

The following requirements are easy to test quickly:

• 1.1: Keyboard
  Test the product with the keyboard. Can you access all items? Do all active items have a highly visible keyboard focus indicator? Can you open all popups, menus, submit forms, etc.?
• 2.2: Pause, Stop, Hide
  Is there movement on the site? Does it have an accessible (mouse, keyboard, touch) pause feature or does it stop within 5 seconds.
• 4.1: Bypass Blocks:
  Does every page have a skip link and is it the first focusable link on the page?
• 3.1: Error identification:
  Submit an empty or incorrect form. Are the errors described appropriately? Do they have suggestions?

If you answered No to any of these questions, check the relevant cell in the VPAT table. Is the issue you found explained there? If not, then you have an inaccurate VPAT. If you have an inaccurate VPAT then you will not know whether the product is accessible or not unless you conduct an accessibility audit yourself.