Creating UI Automation that is Focused, Reliable and Fast

Atomic automated tests: what are they?

One irreducible unit should be formed from an automated test. In other words, your tests should only focus on one thing. This automated test shouldn’t involve end-to-end automation (or anything like it).

Atomic tests only evaluate single features. Aspects of atomic tests include the following:

A test has no more than two assertions. Sometimes, one assertion is necessary to ensure that a state is as it should be.
Atomic tests don’t have very many UI interactions. Also, they are limited to a couple of screens. Atomic tests may navigate through a few screens in some rare cases.

Atomic tests provide solutions for the following problems:

Tests that take too long.
Tests that do not offer the correct failure point.
Unreliable results for automated tests.
Tests difficult to debug.

As a frame of reference, acceptance tests that are automated shouldn’t run for more than thirty seconds when local resources are used.

If a test goes longer then thirty seconds, danger could be on the horizon.

The following is an example of an atomic test:

[Test]
public void ShouldBeAbleToCheckOutWithItems()
{
    //Arrange
    var overviewPage = new CheckoutOverviewPage(Driver);
    overviewPage.Open();
    //We don't need to actually use th UI to add items to the cart. 
    //I'm injecting Javascript to control the state of the cart
    overviewPage.Cart.SetCartState();
    //Act - very few UI interactions
    overviewPage.FinishCheckout().
        IsCheckoutComplete.Should().BeTrue("we finished the checkout process"); //Assert
}

Atomic test advantages

Atomic tests tend to fail quickly

To begin with, having an atomic test written lets you fail early and fast. This suggests you will receive extremely focused feedback fast. It won’t take any longer than 60 seconds to check a feature’s state.

Flaky behavior is decreased through atomic tests

Secondly, atomic tests that are written minimize flakiness, as they reduce the amount of potential breaking points within them. However, flakiness isn’t as much of an issue when integration or unit tests are involved. It remains a big issue with UI automation. Consider the following example:

Look at the homepage of www.ultimateQA.com.
Confirm the opening of the homepage.
Confirm that every section on this page exists.
Have the blog page opened up.
Perform a query for a particular article.
Confirm that said article exists.

When it comes to UI automation, be mindful that something can go wrong during each step. There may have been a change in locators, an interaction mechanism might have been modified, a synchronization strategy could be broken, etc.

As such, when more steps are added, a test has the potential to break, which can subsequently indicate false positives.

Atomic checks create better tests

If a written atomic test fails, it won’t block other functionalities from getting tested. For instance, if the test mentioned above fails the third step, then you may not have an opportunity to see if a blog page is dysfunctional, or if search functionality is operable. Assuming that you don’t have other tests to check this functionality. Test coverage will be reduced because of non-atomic tests.

Atomic tests tend to be quick and brief

Lastly, atomic tests will run faster when parallelized.

Performance enhancement of 98% when testing execution speed by making one change. Here’s how I did it:

Source

I received an average of 98% improvement for test case execution time by conducting parallel atomic tests. Regarding the above scenario, I conducted 18 different end-to-end assessments that weren’t atomic, nor did they run parallel.

With the same code coverage maintained, my tests were broken down into 180 small atomic tests.

They were run in parallel. The average test case time decreased from 86 seconds to 1.76 seconds!

If you want to learn more of such automation best practices as well CI/CD and 60 minute framework development then check out The Complete Selenium WebDriver with Java Bootcamp.

Breaking up large UI end-to-end tests

As you have deduced, atomic tests come with a lot of positives. Still, how can tests running end-to-end be broken up?

This is an issue that many people struggle with. And things get worse: I deal with clients every day that struggle with this very issue. Also, if I could offer a basic solution to this, I would, but I don’t have one.

Culture and technology are key aspects of this challenge for many people. With that said, the guide provided below can be beneficial in conducting atomic tests.

Be mindful that it will not be simple, but when it is achieved, the payoff will be worth it!

Manipulating UI automation test data

Data can be injected using several options, including the following:

RESTful APIs can be used to have the application set in a certain state.
JavaScript can be used.
Data can be injected into a DB in order to have the application set in a specific state.
Use cookies.

If data can be injected between the application’s seams, then each step can be isolated and tested on their own.

Using an API

An API can be used to send out web requests, which will have a user generated.
An API can generate items into an Amazon cart.
Through the use of UI automation, the UI can be opened up to the checkout/cart page.
Test data should be cleaned up afterward.

APIs are extremely fast to use. Web requests can be executed in about 100 milliseconds.

As such, the first two steps – along with the fourth one – won’t take any longer than a second to be executed. The last step remaining is to complete the relevant checkout process.

Things get even better: The use of an API can be a lot more robust in contrast to a UI, as far as test steps go. You can substantially minimize UI automation test flakiness as a result. Here’s an example of a test using an API to control app state:

public void AtomicTest()
{
  //This is an API call that will create a new user
  //Known as the Arrange in unit testing
  var api = new AppApi();
  var testData = api.CreateNewUser();

  //Now we perform our UI interactions
  //The Act in unit testing
  var loginPage = new LoginPage(driver);
  loginPage.Login(testData);
  
  //Now the assertion, the Act
  new ProductsPage(driver).IsLoaded().Should().BeTrue();
  
  //Now clean up, which can also be done in a TearDown hook
  api.DeleteUser(testData.UserId);
}

Using JavaScript to Control an App State

A login screen is a common impediment when it comes to atomic tests. Many of the apps we use have one. How can login screens be removed from tests in order to make them atomic?

Consider the following example:

1. Some JavaScript will be executed with an automation framework as follows:

((IJavaScriptExecutor)_driver).ExecuteScript("window.sessionStorage.setItem('session-username', 'standard-user')");
_driver.Navigate().Refresh();

We are currently logged in, congratulations!

The UI testing tool can be used to conduct the one operation you wish to test.

A complete atomic test may resemble the following:

[Test]
public void ShouldBeAbleToCheckOutWithItems()
{
    //Arrange
    var overviewPage = new CheckoutOverviewPage(Driver);
    overviewPage.Open();
    //We don't need to actually use th UI to add items to the cart. 
    //I'm injecting Javascript to control the state of the cart
    overviewPage.Cart.SetCartState();
    //Act - very few UI interactions
    overviewPage.FinishCheckout().
        IsCheckoutComplete.Should().BeTrue("we finished the checkout process"); //Assert
}

As you can see, this test contains a single UI action, as well as a single assertion.

This is an indication of atomic testing.

Conclusions

I hope that you realized the power of atomic test automation and how it can take your code to the next level. Don’t forget to control the state of your app without a UI so that you can focus on testing the feature that actually matters.