When the Cucumber Scenarios are atomic (having no dependency on each other), there is NO point in running the feature files in parallel for faster execution. The scenarios in all feature file should also be executed to get the maximum execution time reduction.
In this article, we will see how to achieve parallelism in Cucumber using TestNG Framework and learn the differences between JUnit and TestNG.
JUnit vs TestNG
Cucumber supports parallelism with both JUnit and TestNG. There is a difference between them and its a major one. Below we will see the best use cases on when to use JUnit and when to use TestNG.
JUnit
In JUnit, only the feature files run in parallel rather than the scenarios themself. So basically, all the scenarios in a single feature file will be executed by the same single thread. As a test runner, we can either use Maven Surefire or Failsafe plugin.
TestNG
Execution of cucumber scenarios and the rows in scenario outlines is absolutely possible with TestNG. This can be achieved by setting up the dataprovider parallel option as true and extending the runner class with AbstractTestNGCucumberTests from io.cucumber.testng
Let’s get started 🚀
Step 1
Create a simple Maven project called, CucumberParallelScenarios.
Step 2
Add all the necessary dependencies to your maven project:
-
Cucumber-Java
<dependency> <groupId>io.cucumber</groupId> <artifactId>cucumber-java</artifactId> <version>5.0.0-RC1</version> <scope>test</scope> </dependency>
-
TestNG
<dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.14.2</version> <scope>test</scope> </dependency>
-
Cucumber-TestNG
<dependency> <groupId>io.cucumber</groupId> <artifactId>cucumber-testng</artifactId> <version>${cucumber.version}</version> <scope>test</scope> </dependency>
Step 3
Create a new folder under resources named features.
Step 4
Create two feature files and add the below scenarios –
ScenariosExample.feature
Feature: ScenariosExample Example Feature File
Scenario: Number One
Given a step from ‘Scenario 1’ in ‘ScenariosExampleExample’ feature file
Scenario: Number Two
Given a step from ‘Scenario 2’ in ‘ScenariosExampleExample’ feature file
Scenario: Number Three
Given a step from ‘Scenario 3’ in ‘ScenariosExample’ feature file
Scenario: Number Four
Given a step from ‘Scenario 4’ in ‘ScenariosExample’ feature file
Scenario: Number Five
Given a step from ‘Scenario 5’ in ‘ScenariosExample’ feature file
Scenario: Number Six
Given a step from ‘Scenario 6’ in ‘ScenariosExample’ feature file
Scenario: Number Seven
Given a step from ‘Scenario 7’ in ‘ScenariosExample’ feature file
Scenario: Number Eight
Given a step from ‘Scenario 8’ in ‘ScenariosExample’ feature file
Scenario: Number Nine
Given a step from ‘Scenario 9’ in ‘ScenariosExample’ feature file
Scenario: Number Ten
Given a step from ‘Scenario 10’ in ‘ScenariosExample’ feature file
ScenariosOutlineExample.feature
Feature: Outlines Example Feature File Scenario Outline: Multiple Rows Parallel
Given a step from ‘<scenario_outline_row_number>’ in ‘ScenarioOutlinesExample’ feature file
Examples:
| scenario_outline_row_number|
| Scenario Outline – Row 1 |
| Scenario Outline – Row 2 |
| Scenario Outline – Row 3 |
| Scenario Outline – Row 4 |
| Scenario Outline – Row 5 |
Step 5
Now, let’s add the step definition for the above “Given” statement in StepDefs.java file:
import io.cucumber.java.en.Given;
public class StepDefs {
@Given(“a step from {string} in {string} feature file”)
public void step(String scenario, String file) throws InterruptedException {
System.out.format(“Thread ID – %2d – %s from %s feature file.\n”,
Thread.currentThread().getId(), scenario,file);
Thread.sleep(3000);
}
}
Step 6
Add a Test Runner class called – RunParallel.java and add the below snippet:
import org.junit.runner.RunWith;
import org.testng.annotations.DataProvider;
import io.cucumber.junit.Cucumber;
import io.cucumber.testng.AbstractTestNGCucumberTests;
@RunWith(Cucumber.class)
public class RunParallel extends AbstractTestNGCucumberTests {
@Override
@DataProvider(parallel = true)
public Object[][] scenarios() {
return super.scenarios();
}
}
Step 7
Time to run the tests. Now, let’s run the tests by Right Click in RunParallel.java → Run As → TestNG Test. Once done, you will be able to see all the scenarios will be executed in parallel without any delay. The console output will look something like this:
[RemoteTestNG] detected TestNG version 6.14.2
Thread ID – 13 – Scenario Outline – Row 3 from ScenarioOutlinesExample feature file.
Thread ID – 16 – Scenario 1 from ScenariosExampleExample feature file.
Thread ID – 18 – Scenario 3 from ScenariosExample feature file.
Thread ID – 17 – Scenario 2 from ScenariosExampleExample feature file.
Thread ID – 20 – Scenario 5 from ScenariosExample feature file.
Thread ID – 14 – Scenario Outline – Row 4 from ScenarioOutlinesExample feature file.
Thread ID – 19 – Scenario 4 from ScenariosExample feature file.
Thread ID – 11 – Scenario Outline – Row 1 from ScenarioOutlinesExample feature file.
Thread ID – 12 – Scenario Outline – Row 2 from ScenarioOutlinesExample feature file.
Thread ID – 15 – Scenario Outline – Row 5 from ScenarioOutlinesExample feature file.
Thread ID – 19 – Scenario 7 from ScenariosExample feature file.
Thread ID – 18 – Scenario 6 from ScenariosExample feature file.
Thread ID – 17 – Scenario 9 from ScenariosExample feature file.
Thread ID – 20 – Scenario 8 from ScenariosExample feature file.
Thread ID – 12 – Scenario 10 from ScenariosExample feature file.
15 Scenarios ([32m15 passed[0m)
15 Steps ([32m15 passed[0m)
0m6.290s
PASSED: runScenario(“Number Three”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Four”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Five”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Two”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number One”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Seven”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Six”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Nine”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Eight”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
PASSED: runScenario(“Number Ten”, “ScenariosExample Example Feature File”)
Runs Cucumber Scenarios
===============================================
Default test
Tests run: 15, Failures: 0, Skips: 0
=============================================== ===============================================
Default suite
Total tests run: 15, Failures: 0, Skips: 0
===============================================
Code Reference
Code for this article can be found in the GitHub Repository: https://github.com/grajk88/cucumber-scenarios-parallel-example
Conclusion
This is how we achieve parallelism with Cucumber Scenarios which are atomic and have no dependency on each other. The main advantage here is the time reduction as the test execution happens from scenario-level and not from the feature-file level.
hello,
is there any way to execute scenarios parallelly from the feature file using junit and gradle,
if not possible is there any way to reduce the execution time for feature file with multiple independent scenarios