logo logo

Automating Digital Accessibility

Automating Digital Accessibility

In the next iteration of the web content accessibility guidelines (WCAG) the scoring and evaluation system is changing from binary (pass or fail) to a scale system. It will also introduce the concept of ‘Atomic Tests’. 

Atomic tests are used to test both processes and views at the object level. Successful results of the Atomic tests are used to reach a Bronze rating which is the minimum conformance level. Having automated tests can bring you accessibility conformance.

What is digital accessibility? 

It is about making your app or website usable by everyone, no matter how they choose to interact with it 🌐 It could be by using only a keyboard, a screen reader, or any number of other assistive technologies. According to WorldBank.org, one billion people, or 15% of the world’s population, experience some form of disability. It is the only demographic that any one of us could join at any time through accident or illness. As we get older our chances of becoming disabled increase. 

In some ways, making software accessible is supporting your future self 

Alongside those billion people, the Microsoft inclusive design toolkit gave us the concept of situational and temporary disabilities to sit alongside permanent ones. For example; someone who has lost an arm will have the same issues as someone with a broken arm or someone holding a baby. In all cases, only one arm is available. 

Accessibility example

We can also join the temporary set at any time. As someone who has had to use a wheelchair, crutches, and lost use of an arm through a broken elbow (ouch), I can assure you all those make your life so much harder. So, while accessibility is vital to some, it can also make everyone’s experience better. 

Imagine being on a busy train without your headphones. Subtitles allow you to enjoy videos. Imagine your mouse running out of charge. A site that is fully keyboard-accessible enables you to complete all the things you want to do. It goes on…

Not only is it illegal to not be accessible

It could also cost you users, customers, reputation, and revenue. To put that into revenue context, the value of the global disability purchasing market is worth 8 trillion dollars 🤯 That’s not a figure pulled out of the air. That’s from a 2016 report called “The Global Economics of Disability” from a group called “Return On Disability”. That was 5 years ago. It is anyone’s guess if it has increased, but even if not, it is not an insignificant sum. 

What are the web content accessibility guidelines? 

These outline what is required to make your digital products accessible to everyone. From providing a link to skip top-level navigation to keyboard users, to ensuring there is enough contrast so people with low or impaired vision can understand the content. There are currently 78 success criteria with 9 more coming in the new iteration 📝

The web content accessibility guidelines 1.0 (WCAG) were first published in May of 1999. It took nearly a decade for them to be updated to 2.0., and since then 2.1 was released in June 2018. The pace of change is such that it is not sustainable to update them with a decade in between. There has been a commitment to update them regularly, to keep pace with technology. The 3.0 is currently scheduled for the end of the year but with a warning that it could be early 2022 📅 

What are the changes in the new iteration of WCAG? 

The new iteration will have a scale for meeting the success criteria. So instead of passing or failing, it will be possible to meet more of the criteria and even score well if you have a few issues. 

The levels are:

  • Very Poor (0) Any critical errors or less than 50% of related tests pass 
  • Poor (1) No critical errors, approx. 50% to 79% of related tests pass 
  • Fair (2) No critical errors, approx. 80% to 89% of related tests pass 
  • Good (3) No critical errors, approx. 90% to 98% of related tests pass 
  • Excellent (4) No critical errors, approx. 99% to 100% of related tests pass

What’s that got to do with automation? 

Within the new guidelines, there is also the concept of Atomic tests. Atomic tests evaluate the content, most often at an object level for accessibility. Atomic tests include the existing tests that support the existing level A, AA, and AAA success criteria in WCAG 2.1. 

These Atomic tests also include tests that may require additional context or expertise beyond tests that fit within the WCAG 2.1 structure. In WCAG 3.0, Atomic tests are used to test both processes and views. Critical errors within selected processes are also totaled. Successful results of the Atomic tests are used to reach a Bronze rating.

Because these Atomic tests are being designed to be automatable, it is within reason that you can reach conformance to the Bronze level through automation. 

So where do I start?

There are a number of accessibility testing tools that could help you get started in automating tests to meet the Bronze-level success criteria. You don’t have to wait until the new guidelines get published. Your users will thank you for improving your products now 💪

The following tools are free to use: 


WAVE is a suite of evaluation tools that helps authors make their web content more accessible to individuals with disabilities. WAVE can identify many accessibilities and web content accessibility guidelines errors, but also facilitates human evaluation of web content. Their philosophy is to focus on issues that they know impact end-users, facilitate human evaluation, and educate about web accessibility.

The thing I like about WAVE the most is that it shows you the errors, explains what they mean, why it matters and how to fix it. 


Pa11y is a suite of tools that you can use. It’s a command line interface that loads web pages and highlights any accessibility issues it finds. Useful for when you want to run a one-off test against a web page.

We have this tool integrated into our CI (continuous integration) pipeline where I work, and it alerts us if it finds any errors. 


Most experts say you can only automate between 30% to 40% of accessibility tests. The fact that the web content accessibility guidelines are looking to create the next iteration thinking about automating tests as the first step in a great leap forward to help people meet the minimum criteria 😊 

In conclusion, I would be remiss if I didn’t point out that accessibility is a requirement, not a feature!  Just like security, automation can be done, but will only get you so far. Here are a few tips to consider if this is a new subject for you: 

  • If you haven’t acquired a VPAT (Voluntary Product Accessibility Template) for your product(s) that’s a good start
  • If you haven’t got a company accessibility statement- Do it
  • If you haven’t even considered accessibility- Start

There are a lot of good people out there who, for a very reasonable fee, will help you begin your journey. Accessibility is not just about disability, it is very much about inclusion. Thank you for reading ✨

About the author

Ady Stokes

I’m passionate about accessibility, exploring, and testing as part of the creation and development of software. I strongly believe in collaborative methods and using different thought techniques and people’s perspectives to look at things from many angles. Accessibility is about inclusion, not just disability.

Concept creator and lead trainer of Software Testing Bootcamps and Apprenticeships. I have also taught workshops, coached and mentored very successful testers throughout my career.

I have my own blog at The Big Test Theory.com sharing a monthly collection of accessibility news and information called A11y With Ady. It also is the home of my Periodic Table of Testing, a visual heuristic showing the breadth of the testing universe. I have been published in Testing Experience magazine and The Testing Planet. I have given presentations to Agile and Testing groups at meetups, special interest groups and conferences across the UK and Europe. I am very proud to be a founder member and organizer of the Ministry of Testing Leeds meetup.


Twitter: @A11y_Ady

Leave a Reply