In this parallel testing guide, I will explain what Parallel Testing is and then discuss the benefits of parallel testing. After discussing the benefits, we can examine how to get started with Parallel Testing using TestProject.
For years, Sequential Testing was the execution method performed by most teams. Sequential Testing, also known as Serial Testing, is a time-consuming process. It executes each test in a subsequent (coming later) order. Therefore, the next test does not run until the previous test is finished. Parallel Testing is much better and in this article we’ll learn why 😉
Table of Contents – Parallel Testing Guide
- What Is Parallel Testing
- Benefits of Parallel Testing
- Getting Started With Parallel Testing
- Parallel Testing Webinar: Announcing TestProject 3.0
What Is Parallel Testing
Parallel Testing is a test process that concurrently runs multiple tests. We can execute 1 or more tests on multiple browsers with multiple data sets at the same time. The purpose of Parallel Testing is to resolve time limitations by dividing our test across resources. There are many combinations to perform Parallel Testing. For example, we have the ability to test multiple users on Chrome, Firefox, Edge, Safari, and other browser(s).
So, if a Test Suite has 3 users that need to be executed on 3 browsers, then that’s a total of 9 Test Cases. Can you imagine executing 3 usernames/passwords serially on Chrome, Firefox, and Edge? 😵 The job will complete but take a considerable amount of time. However, it won’t take a long time executing the Test Suite via Parallel Testing.
With TestProject, an engineer can create only 1 data-driven Test Script and run it against all devices, browsers, and operating systems (The devices include Android emulators and iOS Simulators). Thus, saving up a bunch of time & effort.
Benefits of Parallel Testing
The main benefit of Parallel Testing is reducing overall execution time. Due to speed, Parallel Testing provides an advantage by delivering fast feedback. Fast feedback helps in several ways such as
- Rapid release cycles
- Quickly pinpoint possible errors
- Improve communication between Software Development Team
Additional Parallel Testing benefits consist of increasing the coverage for testing and optimizing the CI/CD Process.
- Test Coverage is increased by executing an abundant number of platform-browser-devices scenarios in a timely manner.
- CI/CD is optimized by executing Test Cases as soon the developers submit and integrate new code.
Getting Started With Parallel Testing
There are a couple of solutions to get started with Parallel Testing, such as using Selenium Grid or Docker containers (as also described here). Last week, TestProject announced its latest 3.0 version – now becoming another great FREE solution to achieve parallel test execution on a single Agent, without any complex configuration of 3rd party tools (Selenium Grid/Dockers) and without any 3rd party integrations to device farms (Sauce Labs/BrowserStack).
In this parallel testing guide, we will learn how to get started with Parallel Testing by downloading the TestProject Agent. It’s available for our Desktop and Docker. The agent has everything we need to test locally or in the cloud.
❗ Note: The following link provides information about Parallel Execution: Agent Workers Configuration, Parallel Execution, and Parallelization Options.
Create Test
We have the option of creating a Mobile or Web Test. In the test, TestProject allows us to set up parameters and data sets. Parameters are placeholders representing an actual value. A data set is the actual value for executing 1 or more test cases. Here are 2 screenshots showing the different test types and Application Under Test (AUT) – TestProject Example page.
There is a total of 10 steps for this test case. The test case logs into the application, enter/select data then save the data. After saving the data, a verification step confirms Save is visible. 4 of the steps incorporate parameters: Steps 1, 2, 6, and 7
- Step 1 is auto generated for the URL
- Step 2 is a parameter for {Name}
- Step 6 is a parameter for {Address}
- Step 7 is a parameter for {Email}
The value https://example.testproject.io/web/ for ApplicationURL was not added in the .csv file. Each test case uses the same value. However, data sets for Name, Address, and Email are located in the .csv file. The following screenshot displays 5 sets of data that will be used for Data-Driven Testing.
Select Browsers
We can combine and perform Cross Browser Testing with Data-Driven Testing. Both types of testing show why Parallel Testing is important. It’s critical because multiple data sets are executed on multiple browsers. The following screenshot is an example of selecting 3 browsers (Chrome, Edge, and Firefox).
Execute Job
For execution, I created a job called Parallel Testing Job then added 3 Tests. The Data Source called Parallel Testing Data was included so each data set is executed for the parameterized steps.
Serial Execution
When selecting the Execution Settings, we can select Serial or Parallel. Serial executes each test one by one with an option to restart the browser before each test.
While executing the job, the Execution details reveal the process of Serial Testing. 1 browser executes 1 Test Case at a time. Firefox shows up while Chrome and Edge are waiting for execution.
Parallel Execution
TestProject provides 2 ways for executing a Parallel Test.
- Run browsers in parallel with an option to Restart the browser before each test
- Run both browsers & test in parallel
❗ Note: A message at the bottom states “To fully benefit from parallel execution, you should increase the number of workers”. We increase the number of workers by clicking the Optimize button.
The following screenshots display Execution Settings and the details for Parallel Testing. Unlike Serial Testing, all 3 browsers (Firefox, Chrome, & Edge) start executing at the same time.
Compare Serial vs. Parallel Test Executions
After executing the test for Serial and Parallel, the following screenshots reveal a difference in time. The Test Results show Parallel Test executed half the time of Serial Test! ⚡
- The Serial Test executed 3 tests on 3 browsers for a total of 7 minutes and 47 seconds
- The Parallel Test executed 3 tests on 3 browsers for a total of 3 minutes and 51 seconds
Parallel Testing Webinar: Announcing TestProject 3.0
The below webinar video provides even more details on Parallel Testing. Here are some Key Takeaways from the webinar:
- Motivation Behind TestProject 3.0
- Release The Bottleneck
- Serial Execution Method
- Parallel Execution Method
- Selenium Grid vs Docker vs TestProject 3.x
- Demo Parallel Testing For Web
- Demo Parallel Testing For Mobile
- Keep The Version History
- Example CI Flow With TestProject – GitHub
- History Version With TestProject
- Questions & Answers (QA)
Hope you enjoyed this parallel testing guide and would love to hear your own comparison of serial vs. parallel test executions – share it in the comments below 😎