logo logo

Set Your Test Automation Goals

main post image

This tutorial will make web UI testing easy. We will build a simple yet robust web UI test solution using Python, pytest, and Selenium WebDriver. We will learn strategies for good test design as well as patterns for good automation code. By the end of the tutorial, you’ll be a web test automation champ! Your Python test project can be the foundation for your own test cases, too.

Tutorial Chapters

  1. You’re here → Set Your Test Automation Goals (Chapter 1)
  2. Create A Python Test Automation Project Using Pytest (Chapter 2)
  3. Installing Selenium WebDriver Using Python and Chrome (Chapter 3)
  4. Write Your First Web Test Using Selenium WebDriver, Python and Chrome (Chapter 4)
  5. Develop Page Object Selenium Tests Using Python (Chapter 5)
  6. How to Read Config Files in Python Selenium Tests (Chapter 6)
  7. Take Your Python Test Automation To The Next Level (Chapter 7)


Goal-Driven Testing

Many software approaches are “driven” by some sort of objective. Test-driven development makes developers write tests before they write product code. Behavior-driven development encourages teams to define features as specs with examples. I’d like to introduce a new phrase: goal-driven testing.

Before laying a finger on your keyboard to write any tests, ask yourself one question: What are my goals with testing? Testing is a healthy practice, but we don’t test for testing’s sake. Testing is a means to an end: quality. Teams should decide what aspects of quality matter most to them and then enact an appropriate testing strategy. For example, consider some goals and strategies for web app testing:

Goal Strategy
90% of all features have basic test coverage. Use a code coverage tool with unit tests.
Web pages have sub-second page loading times. Use performance testing tools to measure page load times.
Developers are notified within 2 hours of new regressions. Set up continuous testing as part of the delivery pipeline.

Why Include Web UI Testing?

Web apps are designed for human users who interact with pages visually and never see the code underneath. Thus, web UI testing is meant to cover user-like interactions in a real browser. Web UI tests are truly end-to-end because everything behind the front-end site – the services, the databases, and any other systems – must be up and running to support the full app. They answer one big question: Do web app features actually work? They are an indispensable part of a robust web app testing strategy.

Thankfully, web UI tests can be automated for consistency, repeatability, and speed. Packages like Selenium WebDriver enable programmers to automate browser interactions with languages like Java, C#, and Python. More advanced tools like TestProject provide a full platform out of the box for recording, refactoring, and reporting web UI tests.

Web UI testing has drawbacks, though:

  1. Web UI tests are slow, even when automated. They need time to open a browser, load pages, and send interactions. A typical automated web UI test takes about a minute to complete, whereas a typical unit test takes a millisecond.
  2. Web UI tests cannot pinpoint failures in the code like unit tests can. For example, a web UI test could reveal where string formats are displayed incorrectly, but it cannot point to the specific method, file, and line number at fault.
  3. Web UI tests can be flaky. Any black box test relies upon the system under test. Race conditions and halting problems are simply to be expected. Test automation code needs careful design and safety checks to become robust.

In short, web UI tests are expensive: they require time, expertise, and maintenance.

How To Make It Worthwhile

The best way to mitigate these drawbacks is to balance the testing strategy with return on investment. Not every feature should have a Web UI test. Instead, focus on the most critical features, and stick to “happy” paths rather than edge cases. Make sure to run solid coverage with tests at the unit and integration levels, too, to catch bugs early. More coverage at lower layers means less risk at higher layers, which in turn means fewer web UI tests are needed.

This tradeoff is best illustrated by the renowned Testing Pyramid:

The Testing Pyramid
The Testing Pyramid

The Testing Pyramid encourages teams to balance high-level, expensive tests (like web UI tests) with more low-level, fast, simple tests (like unit tests or web service API tests).

Our Goals

Our goals for this tutorial are straightforward:

  1. Learn the basics of web UI test automation with Python.
  2. Set up a good Python test automation project that can be extended with new tests.
  3. Write and run a few example tests locally.

As such, we will focus exclusively on web UI testing.

If you’re ready to start coding, then jump into the next chapter!

 

TestProject Test Automation Tool

Avatar

About the author

AutomationPanda

Andy Knight is the "Automation Panda" - an engineer, consultant, and international speaker who loves all things software. He specializes in building robust test automation systems from the ground up. Read his tech blog at AutomationPanda.com, and follow him on Twitter at @AutomationPanda.

Join TestProject Community

Get full access to the world's first cloud-based, open source friendly testing community. Enjoy TestProject's end-to-end test automation Platform, Forum, Blog and Docs - All for FREE.

Join Us Now

Leave a Reply

Join TestProject Newsletter

Join a 20K community of readers! Always stay up-to-date with all the latest test automation trends, best practice and tips shared by leading software testing community experts across the globe!

FacebookLinkedInTwitterEmail