This tutorial will make web UI testing easy. We will build a simple yet robust web UI test solution using Python, pytest, and Selenium WebDriver. We will learn strategies for good test design as well as patterns for good automation code. By the end of the tutorial, you’ll be a web test automation champ! Your Python test project can be the foundation for your own test cases, too.
📍 If you are looking for a single Python Package for Android, iOS and Web Testing – there is also an easy open source solution provided by TestProject. With a single executable, zero configurations, and familiar Selenium APIs, you can develop and execute robust Python tests and get automatic HTML test reports as a bonus! All you need is: pip install testproject-python-sdk. Simply follow this Github link to learn more about it, or read through this great tutorial to get started.
Tutorial Chapters
- Web UI Testing Made Easy with Python, Pytest and Selenium WebDriver (Overview)
- Set Your Test Automation Goals (Chapter 1)
- You’re here → Create A Python Test Automation Project Using Pytest (Chapter 2)
- Installing Selenium WebDriver Using Python and Chrome (Chapter 3)
- Write Your First Web Test Using Selenium WebDriver, Python and Chrome (Chapter 4)
- Develop Page Object Selenium Tests Using Python (Chapter 5)
- How to Read Config Files in Python Selenium Tests (Chapter 6)
- Take Your Python Test Automation To The Next Level (Chapter 7)
- Create Pytest HTML Test Reports (Chapter 7.1)
- Parallel Test Execution with Pytest (Chapter 7.2)
- Scale Your Test Automation using Selenium Grid and Remote WebDrivers (Chapter 7.3)
- Test Automation for Mobile Apps using Appium and Python (Chapter 7.4)
- Create Behavior-Driven Python Tests using Pytest-BDD (Chapter 7.5)
Now that we know why we should do web UI testing and what our goals should be, let’s set up a Python test automation project using pytest!
Why Python?
Python is one of the most popular programming languages currently available. It powers web backends, data science notebooks, sysadmin scripts, and more. Its syntax is clean, readable, and elegant – perfect for both beginners and experts. And everything you could imagine is just an import away. Naturally, Python is also a great language for test automation. Its conciseness lets testers focus more on the test and less on the code. Testers who haven’t done much programming tend to learn Python faster than other languages like Java or C#. Python is perfect for kickstarting tests!
What is pytest?
At the heart of any functional test automation project is the “core” test framework. The framework handles test case structure, test execution, and pass/fail result reporting. It is the foundation upon which extra packages and code (like Selenium WebDriver) can be added.
pytest is one of Python’s best test frameworks. It is simple, scalable, and Pythonic. Test cases are written as functions, not classes. Test assertion failures are reported with actual values. Plugins can add code coverage, pretty reports, and parallel execution. pytest can integrate with other frameworks like Django and Flask, too. According to the Python Developers Survey 2018, it is also the most popular Python test framework.
Getting Started
Let’s create our Python test project! If you haven’t already done so, please download and install Python 3 on your machine. Then, create a new directory for the project:
$ mkdir python-webui-testing $ cd python-webui-testing
Whenever I create a new Python project, I create a virtual environment for its dependencies. That way, projects on the same machine won’t have conflicting package versions. I use pipenv because it simplifies the workflow. To install pipenv globally, run:
$ pip install pipenv
Then, install pytest for your new project:
$ pipenv install pytest --dev
Pipenv will add two new files to your project: Pipfile and Pipfile.lock. Pipfile specifies the project’s requirements, whereas Pipfile.lock “locks” the explicit versions that the project will use. The “–dev” option in the command denotes that the pytest package will be used only for development and not for deployment.
The First Test
By convention, most projects put all tests under a tests/ directory. Let’s follow this convention:
$ mkdir tests $ cd tests
Create a Python module named test_math.py for our first test, and add the following code:
def test_addition(): assert 1 + 1 == 2
Tests written using pytest typically don’t need much code. These two lines are a fully functional test case! The test case is written as a function, not as a class. Imports are unnecessary for a basic test like this. Python’s native assert statement is used instead of custom assertion calls.
Running Tests
Let’s run our new test. Change directory back to the project root, and invoke the pytest module:
$ cd .. $ pipenv run python -m pytest ============================= test session starts ============================== platform darwin -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.12.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-webui-testing collected 1 item tests/test_math.py . [100%] =========================== 1 passed in 0.02 seconds ===========================
Our first test passed!
How did pytest discover our test? By name: pytest will search for test functions named test_* in modules named test_*.py. Interestingly, pytest doesn’t need a __init__.py file in any test directory.
Failed Tests
What happens if a test fails? Let’s add another test with a bug to find out:
def test_subtraction(): diff = 1 - 1 assert diff == 1
Now, when we run it, we see this:
$ pipenv run python -m pytest
============================= test session starts ==============================
platform darwin -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
rootdir: /Users/andylpk247/Programming/automation-panda/python-webui-testing
collected 2 items
tests/test_math.py .F [100%]
=================================== FAILURES ===================================
_______________________________ test_subtraction _______________________________
def test_subtraction():
diff = 1 - 1
> assert diff == 1
E assert 0 == 1
tests/test_math.py:13: AssertionError
====================== 1 failed, 1 passed in 0.08 seconds ======================
The test_subtraction test fails with “F” instead of “.”. Furthermore, pytest prints trace messages showing the failed assertion with the module and line number. Notice that the actual values for each expression in the assertion are printed: diff is evaluated to 0, which is clearly not 1. Cool! This assertion introspection is very helpful when analyzing test failures.
Let’s fix that bug:
def test_subtraction(): diff = 1 - 1 assert diff == 0
And let’s rerun those tests:
$ pipenv run python -m pytest ============================= test session starts ============================== platform darwin -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.12.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-webui-testing collected 2 items tests/test_math.py .. [100%] =========================== 2 passed in 0.02 seconds ===========================
We’re back on track.
Parametrized Tests
What if we want to run the same test procedure with multiple input combos? pytest has a decorator for that! Let’s write a new test for multiplication with parametrized inputs:
import pytest @pytest.mark.parametrize( "a,b,expected", [(0, 5, 0), (1, 5, 5), (2, 5, 10), (-3, 5, -15), (-4, -5, 20)]) def test_multiplication(a, b, expected): assert a * b == expected
This time, the pytest module must be imported. The @pytest.mark.parametrize decorator will substitute tuples of inputs for test function arguments, running the test function once per input tuple. Running the tests again will show many more passing dots:
$ pipenv run python -m pytest ============================= test session starts ============================== platform darwin -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.12.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-webui-testing collected 7 items tests/test_math.py ....... [100%] =========================== 7 passed in 0.03 seconds ===========================
Sweet! Parameters are a great way to do data-driven testing.
Verifying Exceptions
pytest treats unhandled exceptions as test failures. In fact, the assert statement simply throws an exception to register a failure. What if we want to verify that an exception is correctly raised? Use pytest.raises with the desired exception type, like this:
def test_divide_by_zero():
with pytest.raises(ZeroDivisionError):
1 / 0
Rerun the tests to make sure all is well:
$ pipenv run python -m pytest ============================= test session starts ============================== platform darwin -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.12.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-webui-testing collected 8 items tests/test_math.py ........ [100%] =========================== 8 passed in 0.04 seconds ===========================
Nice. Math still works! 😉
More Info
We covered the fundamentals in this chapter, but pytest has much more to offer. Check out pytest’s official site to learn more advanced features of the framework. We’ll see how to use fixtures and a few plugins later in this tutorial, too.
In the next chapter, we’ll learn how pytest can do web UI testing with Selenium WebDriver.