logo logo

Parallel Testing Guide using TestProject

Parallel Testing Guide using TestProject

In this parallel testing guide, I will explain what Parallel Testing is and then discuss the benefits of parallel testing. After discussing the benefits, we can examine how to get started with Parallel Testing using TestProject.

For years, Sequential Testing was the execution method performed by most teams. Sequential Testing, also known as Serial Testing, is a time-consuming process. It executes each test in a subsequent (coming later) order. Therefore, the next test does not run until the previous test is finished. Parallel Testing is much better and in this article we’ll learn why 😉

Table of Contents – Parallel Testing Guide

What Is Parallel Testing

Parallel Testing is a test process that concurrently runs multiple tests. We can execute 1 or more tests on multiple browsers with multiple data sets at the same time. The purpose of Parallel Testing is to resolve time limitations by dividing our test across resources. There are many combinations to perform Parallel Testing. For example, we have the ability to test multiple users on Chrome, Firefox, Edge, Safari, and other browser(s).

So, if a Test Suite has 3 users that need to be executed on 3 browsers, then that’s a total of 9 Test Cases. Can you imagine executing 3 usernames/passwords serially on Chrome, Firefox, and Edge? 😵 The job will complete but take a considerable amount of time. However, it won’t take a long time executing the Test Suite via Parallel Testing.

With TestProject, an engineer can create only 1 data-driven Test Script and run it against all devices, browsers, and operating systems (The devices include Android emulators and iOS Simulators). Thus, saving up a bunch of time & effort.

Benefits of Parallel Testing

The main benefit of Parallel Testing is reducing overall execution time. Due to speed, Parallel Testing provides an advantage by delivering fast feedback. Fast feedback helps in several ways such as

  • Rapid release cycles
  • Quickly pinpoint possible errors
  • Improve communication between Software Development Team

Additional Parallel Testing benefits consist of increasing the coverage for testing and optimizing the CI/CD Process.

  • Test Coverage is increased by executing an abundant number of platform-browser-devices scenarios in a timely manner.
  • CI/CD is optimized by executing Test Cases as soon the developers submit and integrate new code.

Getting Started With Parallel Testing

There are a couple of solutions to get started with Parallel Testing, such as using Selenium Grid or Docker containers (as also described here). Last week, TestProject announced its latest 3.0 version – now becoming another great FREE solution to achieve parallel test execution on a single Agent, without any complex configuration of 3rd party tools (Selenium Grid/Dockers) and without any 3rd party integrations to device farms (Sauce Labs/BrowserStack).

In this parallel testing guide, we will learn how to get started with Parallel Testing by downloading the TestProject Agent. It’s available for our Desktop and Docker. The agent has everything we need to test locally or in the cloud.

Download TestProject Agent

Note: The following link provides information about Parallel Execution: Agent Workers Configuration, Parallel Execution, and Parallelization Options.

Create Test

We have the option of creating a Mobile or Web Test. In the test, TestProject allows us to set up parameters and data sets. Parameters are placeholders representing an actual value. A data set is the actual value for executing 1 or more test cases. Here are 2 screenshots showing the different test types and Application Under Test (AUT) – TestProject Example page.

TestProject - Create New Test
TestProject Example Page

Add Parameters

There is a total of 10 steps for this test case. The test case logs into the application, enter/select data then save the data. After saving the data, a verification step confirms Save is visible. 4 of the steps incorporate parameters: Steps 1, 2, 6, and 7

  • Step 1 is auto generated for the URL
  • Step 2 is a parameter for {Name}
  • Step 6 is a parameter for {Address}
  • Step 7 is a parameter for {Email}

TestProject Codeless Test Case
TestProject Codeless Test Case_Parameters

Add Data Sets

The value https://example.testproject.io/web/ for ApplicationURL was not added in the .csv file. Each test case uses the same value. However, data sets for Name, Address, and Email are located in the .csv file. The following screenshot displays 5 sets of data that will be used for Data-Driven Testing.

Data Driven Testing with TestProject

Select Browsers

We can combine and perform Cross Browser Testing with Data-Driven Testing. Both types of testing show why Parallel Testing is important. It’s critical because multiple data sets are executed on multiple browsers. The following screenshot is an example of selecting 3 browsers (Chrome, Edge, and Firefox).

Use TestProject to Combine Cross Browser Testing with Data-Driven Testing

Execute Job

For execution, I created a job called Parallel Testing Job then added 3 Tests. The Data Source called Parallel Testing Data was included so each data set is executed for the parameterized steps.

TestProject Parallel Testing Job
TestProject Parallel Testing Data

Serial Execution

When selecting the Execution Settings, we can select Serial or Parallel. Serial executes each test one by one with an option to restart the browser before each test.

TestProject Serial Execution

While executing the job, the Execution details reveal the process of Serial Testing. 1 browser executes 1 Test Case at a time. Firefox shows up while Chrome and Edge are waiting for execution.

TestProject Serial Execution

Parallel Execution

TestProject provides 2 ways for executing a Parallel Test.

  1. Run browsers in parallel with an option to Restart the browser before each test
  2. Run both browsers & test in parallel

Note: A message at the bottom states “To fully benefit from parallel execution, you should increase the number of workers”. We increase the number of workers by clicking the Optimize button.

The following screenshots display Execution Settings and the details for Parallel Testing. Unlike Serial Testing, all 3 browsers (Firefox, Chrome, & Edge) start executing at the same time.

TestProject Parallel Testing Execution Settings

TestProject Parallel Testing Execution

Compare Serial vs. Parallel Test Executions

After executing the test for Serial and Parallel, the following screenshots reveal a difference in time. The Test Results show Parallel Test executed half the time of Serial Test! ⚡

  • The Serial Test executed 3 tests on 3 browsers for a total of 7 minutes and 47 seconds
  • The Parallel Test executed 3 tests on 3 browsers for a total of 3 minutes and 51 seconds

Compare Serial vs. Parallel Test Executions
Compare Serial vs. Parallel Test Executions

Parallel Testing Webinar: Announcing TestProject 3.0

The below webinar video provides even more details on Parallel Testing. Here are some Key Takeaways from the webinar:

  • Motivation Behind TestProject 3.0
  • Release The Bottleneck
  • Serial Execution Method
  • Parallel Execution Method
  • Selenium Grid vs Docker vs TestProject 3.x
  • Demo Parallel Testing For Web
  • Demo Parallel Testing For Mobile
  • Keep The Version History
  • Example CI Flow With TestProject – GitHub
  • History Version With TestProject
  • Questions & Answers (QA)

Hope you enjoyed this parallel testing guide and would love to hear your own comparison of serial vs. parallel test executions – share it in the comments below 😎

About the author

Rex Jones II

Rex Jones II has a passion for sharing knowledge about testing software. His background is development but enjoys testing applications.

Rex is an author, trainer, consultant, and former Board of Director for User Group: Dallas / Fort Worth Mercury User Group (DFWMUG) and member of User Group: Dallas / Fort Worth Quality Assurance Association (DFWQAA). In addition, he is a Certified Software Tester Engineer (CSTE) and has a Test Management Approach (TMap) certification.

Recently, Rex created a social network that demonstrate automation videos. In addition to the social network, he has written 6 Programming / Automation books covering VBScript the programming language for QTP/UFT, Java, Selenium WebDriver, and TestNG.

✔️ YouTube https://www.youtube.com/c/RexJonesII/videos
✔️ Facebook http://facebook.com/JonesRexII
✔️ Twitter https://twitter.com/RexJonesII
✔️ GitHub https://github.com/RexJonesII/Free-Videos
✔️ LinkedIn https://www.linkedin.com/in/rexjones34/

Leave a Reply