The last session of the Trusted Tester Study Group covered both testing and conformance reporting. In part 1 we’ve already covered Topic 1: Conforming Alternate Version and Non-Interference. In part 2, we will find out how to best package our test findings to ship them off.
You can watch parts 1 and 2 on YouTube.
So as now we are through with all the test IDs, it is time to write our testing report! At first this may seem intimidating, but if you completed all 20 topics in this course, trust me, you have already accomplished harder things!
Tester name (and Trusted Tester ID for Section 508 compliance)
Operating system and browser used
Version and date of test
URL(s) tested
Scope of testing (full site, a specific user flow, section, application)
Results for each test ID (pass/fail/does not apply/not tested)
Description of failures with the exact location
Steps to reproduce the issues
Screenshots (when applicable) are highly recommended, but not mandatory.
There is no specific format you must use. As long as the above are included, you can use any format that you are comfortable with. That could be a written report, a spreadsheet or a presentation deck.
For an extensive, formal report, it tends to be a written one. I know, I know! Writing a full report with who knows how many pages sounds intimidating at first. But rejoice! You don’t have to write it from scratch. There are templates you can use:
This reporting tool was developed by DHS specifically for use with the Trusted Tester test process. It has the advantage of merging results from different test conditions to determine if specific WCAG Success Criteria are met. ACRT also provides some conditional logic and quality assurance checks.
The Information Technology Industry Council provides the global standard for the Voluntary Product Accessibility Template. There are versions for International Reporting (INT version) and for European standards (EU version). The version that is most commonly used in the United States is the Web Content Accessibility Guidelines (WCAG) VPAT.
Consider this one as more of an honorable mention, as it is not from the course, but my own recommendation. The Assessment option walks you through the test process, measuring compliance with WCAG 2.2 Level AA success criteria. As we’ve learned in Part 10: Synchronized Media, the Trusted Tester includes some non-WCAG based test IDs, so this one may not be the best fit if you need to put your Trusted Tester ID on it!
These three are trusty companions on all your auditing quests! Once you feel familiar with the report structure, you can make a custom template to use for your audits. Just don’t feel as if you need to have the perfect template ready when you’re just getting started. You can’t use level 10 spells at level 4, so stick to the reliable tricks first (Eldritch Blast).
Beyond the test IDs, the Pass, Fail and Does-Not-Applies, the real juicy part of a report (any report if we are honest) are the comments! Here, the reader finds out WTF is going on.
A good explanation is detailed and tells the reader exactly where the problem is. It should not be a puzzle to solve. Rather, it should give the reader all the clues to solve the issue: Where it happens, what it is or does, how it fails, and any specific details needed to reproduce it in an isolated test.
“On the Credit Card Payment screen (URL), users are not permitted an option to confirm their order information before submitting. There’s also no option to check or reverse the order. Further confirmation page with summary order information is not provided for credit card orders.”
We have the location: Credit Card Payment screen + URL
We learn how it fails: no option to confirm the order, check or reverse it,
And that this is specific to the credit card orders: “not provided for credit card orders.”
“On the Enter Time page, the Add button fails.”
This description lacks the explanation why or rather how it fails. In the case of a button, there could be many reasons! Color contrast, label, keyboard focus… (I could go on, but I think you catch my drift)
We don’t know what the button does instead. Does it do anything?
We don’t have the steps to reproduce the failure. We have no clue how we get to this failure!
Now you are equipped to write a conformance report to end all accessibility issues. So go forth on your quest to defeat digital access barriers!
ANDI Accessibility Testing Tool (bookmarklet)
US Access Board: Revised Section 508 Standards
If you have questions, post them to the GDG event forum or submit as GitHub issues.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.