Why testing every website maybe be unnecessary
Video transcript
What do you do when you have a large number of websites to assess for accessibility? Do you check everything, check some or somewhere in between? This is a situation we regularly encounter and this is how we tackle it.
One of the challenges we regularly see is assessing the accessibility across multiple sites. An organisation might have 10, 20 or even 50 sites, and their initial instinct is to test everything.
But that can mean spending a lot of time and money identifying the same problems over and over again. A better question to ask is how different are those sites?
Because they might have different branding, different content or even a different domain. But underneath, use the same templates and potentially the same code. If that's the case, it may only be a smaller number of assessments required as the same codebase repeats across those sites.
How we approach it is breaking down that large number of sites into a representative selection and assessing those first. The goal is trying to learn something about the entire collection of sites.
We take a cross section of small, medium and large sites. And use the WCAG Evaluation Methodology to then find a representative sample of pages to test.
The methodology helps identify the scope, different page types and components and works out what needs testing. The aim is to select a variety of types of content that best represent each site.
Once we have the sample pages, automated and manual testing is applied. Automation quickly identifies straightforward issues and show areas that need investigation, but it can't understand everything happening in an interface. Manual assessment is crucial. We inspect the HTML, navigate with the keyboard and look at what's exposed through the browser's accessibility tree. All this finds issues far beyond what an automated tool can.
This is where assessing sites in stages becomes valuable. Once the first group of sites has been assessed, we bring the findings together and look across the entire collection of sites. Which issues appear repeatedly? Which templates and components are shared and which problems could be fixed centrally?
That analysis helps inform what the next assessment should look like. We might find substantial commonality across the collection. In that case, our advice is to look at fixing shared components and then validate their implementation, this can sometimes be a much more efficient approach.
Alternatively, we might find that all the websites are more diverse than expected. And recommend expanding the testing with further assessments. The important thing is the decision and next steps are based on evidence rather than assumption.
Retesting matters, findings aren't closed when it's marked as fixed. They need to be retested, by reproducing the scenario and verifying the fix actually works.
We can see what was identified in the initial assessment, what was supposed to change and whether the change was successful, all incredibly valuable for accountability purposes. But accessibility doesn't finish with the assessment. Websites, content and components continue to change and evolve.
Eventually your organisation needs to move away from being reactive with accessibility and move towards ongoing management.
That might include regular assessments, accessibility assurance, conformance reports, accessibility statements and accessibility pulse checks becoming part of your digital delivery.
The long-term objective isn't to have more accessibility assessments. It's reducing new accessibility problems from being introduced or regressing in the first place.
If you're responsible for accessibility across a large portfolio of websites, don't assume the approach is to assess everything.
First understand the how large your site collection is. Then identify representative sites, pages, components and functionality within them.
Look for shared components and recurring accessibility barriers. Then use the evidence from the first stage assessments to determine whether deeper individual assessments are needed.
This gives you a roadmap for improving accessibility across your entire digital estate, which is far more useful than a collection of disconnected accessibility reports.
We have a lot of experience with large sites and managing the complexity with accessibility. If you have some of the same challenges, reach out and let's see if we can help you.
As always folks if you've enjoyed this video and want to get more from the best channel for digital accessibility make sure you've subscribed to CANAXESS.
Until next time, see you in the next video.