The advantages of using the TestProject OpenSDK for Python have been widely documented since its release, but did you know you can integrate aXe accessibility tests with TestProject and take advantage of the provided Reporting Dashboard?
In this article, we will look at the very simple steps required to report aXe test results in TestProject. But before we get started, let’s look at aXe.
aXe is a web accessibility testing tool that validates web pages against the Web Content Accessibility Guidelines, a list of accessibility standards that web & mobile applications should abide by ensuring accessibility to the many millions who suffer from disabilities. aXe can be used as a browser plugin but there are also many open source packages/libraries which enable testers to incorporate aXe tests into their automation suites alongside browser automation tools such as Selenium. In this demo, we’ll be using aXe with Python.
Setup
First, we’ll need to install the following packages
- TestProject Python SDK (pip install testproject-python-sdk)
- Pytest (pip install pytest)
- aXe (pip install axe-selenium-python)
In addition to installing these packages, we also need to add our TestProject Dev Token as an environment variable inside our project. The token can be retrieved from the Integrations tab of the TestProject portal.
Writing the Test
Now that the setup is done, we can jump straight into the script. For the purposes of this tutorial, we’ll be using a very simple script that simply reports any accessibility issues flagged on the registration page of this test todo application.
The first test steps
We can use Selenium to navigate to the page in Chrome and then we can inject aXe to check the accessibility. This will produce a report detailing all recommendations based on the criteria set out in the Web Content Accessibility Guidelines (WCAG).
We will be adding an assertion that will set the test to passed or failed depending on the number of accessibility issues found. If none, the test will pass, however, if one or more issues are identified, the test will be marked as failed.
To start, let’s create a new Python file in our project called test_accessibility.py. Now inside our file, we can add our import statements. We only need 3:
from src.testproject.sdk.drivers import webdriver from src.testproject.decorator import report_assertion_errors from axe_selenium_python import Axe
The first imports the TestProject Python SDK, and the second one allows us to record the status of the test based on the result of our assert statement. The last allows us to run our accessibility test using aXe.
Next, we can define our test function and navigate to our application under test. If you know selenium, you’ll be familiar with the following steps:
def test_accessibility():
driver = webdriver.Chrome()
driver.get("https://flask-todo-test.herokuapp.com/register")
OK, great, this will get us to our registration page. Now to run aXe. The code we’re using here is taken direct from the usage section of axe-selenium-python on PyPi. It’s just 3 lines of code and its super straight forward.
axe = Axe(driver) axe.inject() results = axe.run()
In this code we initialize aXe, inject axe-core into the webpage and then run it, storing the output in a variable named results.
If you want to see what the output looks like you can add the following line to your test:
axe.write_results(results, 'results.json')
This will output the accessibility results to a json file inside your project directory. You’ll see that the json file is split into sections. The one we’re interested in is violations. This contains the list of accessibility issues aXe has identified.
Adding the validations
So far, we’ve navigated to our page and run our accessibility test, now we need to add our assertion and report the results in TestProject. In our assertion we’re going to count the number of ‘violations’ reported in the aXe results. This can be done with the following line of code
assert len(results["violations"]) == 0, axe.report(results["violations"])
The first part, before the comma, checks if the number of violations in the results is equal to zero. If not zero then the assertion will fail. The second part will output the violations in the failure report.
Now, if we left our script as it is our test will always show as passed in TestProject, even if there were violations. To make sure assertion failures are correctly reported in TestProject, we need to add the following decorator above our test function, like so:
@report_assertion_errors def test_accessibility():
Cool, almost there 👌 We just need to quit the browser, this is important as it is how TestProject knows to record the test results.
driver.quit()
OK – so our full test should look like this:
from src.testproject.sdk.drivers import webdriver
from axe_selenium_python import Axe
from src.testproject.decorator import report_assertion_errors
@report_assertion_errors
def test_accessibility():
# Navigate to Application
driver = webdriver.Chrome()
driver.get("https://flask-todo-test.herokuapp.com/register")
# run axe
axe = Axe(driver)
axe.inject()
results = axe.run()
# Output results to file - optional line
axe.write_results(results, 'results.json')
# Check the number of violations is zero. If not zero fail assertion
assert len(results["violations"]) == 0, axe.report(results["violations"])
# Quit Driver
driver.quit()
Running the Test
Now that we have our test, let’s run it using pytest. From inside our project, we can just call pytest via the terminal.
The results in the terminal show that the test has failed and that 4 accessibility violations have been reported:
Now let’s switch over to the TestProject Platform to see what it looks like there on the reporting dashboards. If we hover over the reports tab we can see that our test has failed:
If we drill down into this test, we can see which step has failed:
Clicking on the failed step will also show the violation details. This is not formatted, but you can always go back and consult the results.json file which gets output in the Python project directory.
Summary
And there you have it: Accessibility test results in TestProject, in just a few lines of code. Good job! 💪 The TestProject OpenSDK supports the pytest and unittest frameworks in Python, so anything you can do inside these frameworks with Selenium can be reported here. It’s a great way of keeping your results together and utilizing the power of TestProject reporting 📊
Well, that’s a wrap so as always Happy Testing! 😎