Running Cucumber Scenarios in Parallel & NOT Feature Files

When the Cucumber Scenarios are atomic (having no dependency on each other), there is NO point in running the feature files in parallel for faster execution. The scenarios in all feature file should also be executed to get the maximum execution time reduction.

In this article, we will see how to achieve parallelism in Cucumber using TestNG Framework and learn the differences between JUnit and TestNG.

JUnit vs TestNG

Cucumber supports parallelism with both JUnit and TestNG. There is a difference between them and its a major one. Below we will see the best use cases on when to use JUnit and when to use TestNG.

JUnit

In JUnit, only the feature files run in parallel rather than the scenarios themself. So basically, all the scenarios in a single feature file will be executed by the same single thread. As a test runner, we can either use Maven Surefire or Failsafe plugin.

TestNG

Execution of cucumber scenarios and the rows in scenario outlines is absolutely possible with TestNG. This can be achieved by setting up the dataprovider parallel option as true and extending the runner class with AbstractTestNGCucumberTests from io.cucumber.testng

 

Let’s get started 🚀

Step 1

Create a simple Maven project called, CucumberParallelScenarios.

Step 2

Add all the necessary dependencies to your maven project:

• Cucumber-Java

<dependency>
 	   <groupId>io.cucumber</groupId>
     	  <artifactId>cucumber-java</artifactId>
  	  <version>5.0.0-RC1</version>
 	   <scope>test</scope>
</dependency>

• TestNG

<dependency>
<groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>6.14.2</version>
  <scope>test</scope>
</dependency>

• Cucumber-TestNG

<dependency>
  <groupId>io.cucumber</groupId>
  <artifactId>cucumber-testng</artifactId>
  <version>${cucumber.version}</version>
  <scope>test</scope>
</dependency>

Step 3

Create a new folder under resources named features.

Step 4

Create two feature files and add the below scenarios –

ScenariosExample.feature

     Feature: ScenariosExample Example Feature File

     Scenario: Number One 
         Given a step from ‘Scenario 1’ in ‘ScenariosExampleExample’ feature file 

     Scenario: Number Two 
         Given a step from ‘Scenario 2’ in ‘ScenariosExampleExample’ feature file 

     Scenario: Number Three 
         Given a step from ‘Scenario 3’ in ‘ScenariosExample’ feature file       

     Scenario: Number Four 
         Given a step from ‘Scenario 4’ in ‘ScenariosExample’ feature file       

     Scenario: Number Five 
         Given a step from ‘Scenario 5’ in ‘ScenariosExample’ feature file       

     Scenario: Number Six 
         Given a step from ‘Scenario 6’ in ‘ScenariosExample’ feature file       

     Scenario: Number Seven 
         Given a step from ‘Scenario 7’ in ‘ScenariosExample’ feature file       

     Scenario: Number Eight 
         Given a step from ‘Scenario 8’ in ‘ScenariosExample’ feature file       

     Scenario: Number Nine 
         Given a step from ‘Scenario 9’ in ‘ScenariosExample’ feature file       

     Scenario: Number Ten 
         Given a step from ‘Scenario 10’ in ‘ScenariosExample’ feature file

ScenariosOutlineExample.feature

Feature: Outlines Example Feature File   Scenario Outline: Multiple Rows Parallel

    Given a step from ‘<scenario_outline_row_number>’ in ‘ScenarioOutlinesExample’ feature file

    Examples: 
      | scenario_outline_row_number|
      | Scenario Outline – Row 1 |
      | Scenario Outline – Row 2 |
      | Scenario Outline – Row 3 |
      | Scenario Outline – Row 4 |
      | Scenario Outline – Row 5 |

 

Step 5

Now, let’s add the step definition for the above “Given” statement in StepDefs.java file:

import io.cucumber.java.en.Given;

public class StepDefs {

    @Given(“a step from {string} in {string} feature file”)
    public void step(String scenario, String file) throws InterruptedException {

        System.out.format(“Thread ID – %2d – %s from %s feature file.\n”,

        Thread.currentThread().getId(), scenario,file);

        Thread.sleep(3000);
    }
}

 

Step 6

Add a Test Runner class called – RunParallel.java and add the below snippet:

import org.junit.runner.RunWith;
import org.testng.annotations.DataProvider;

import io.cucumber.junit.Cucumber;
import io.cucumber.testng.AbstractTestNGCucumberTests;

@RunWith(Cucumber.class)
public class RunParallel extends AbstractTestNGCucumberTests {

      @Override
      @DataProvider(parallel = true)
      public Object[][] scenarios() {
            return super.scenarios();
      }
}

Step 7

Time to run the tests. Now, let’s run the tests by Right Click in RunParallel.java → Run As → TestNG Test. Once done, you will be able to see all the scenarios will be executed in parallel without any delay. The console output will look something like this:

[RemoteTestNG] detected TestNG version 6.14.2
Thread ID – 13 – Scenario Outline – Row 3 from ScenarioOutlinesExample feature file.
Thread ID – 16 – Scenario 1 from ScenariosExampleExample feature file.
Thread ID – 18 – Scenario 3 from ScenariosExample feature file.
Thread ID – 17 – Scenario 2 from ScenariosExampleExample feature file.
Thread ID – 20 – Scenario 5 from ScenariosExample feature file.
Thread ID – 14 – Scenario Outline – Row 4 from ScenarioOutlinesExample feature file.
Thread ID – 19 – Scenario 4 from ScenariosExample feature file.
Thread ID – 11 – Scenario Outline – Row 1 from ScenarioOutlinesExample feature file.
Thread ID – 12 – Scenario Outline – Row 2 from ScenarioOutlinesExample feature file.
Thread ID – 15 – Scenario Outline – Row 5 from ScenarioOutlinesExample feature file.
Thread ID – 19 – Scenario 7 from ScenariosExample feature file.
Thread ID – 18 – Scenario 6 from ScenariosExample feature file.
Thread ID – 17 – Scenario 9 from ScenariosExample feature file.
Thread ID – 20 – Scenario 8 from ScenariosExample feature file.
Thread ID – 12 – Scenario 10 from ScenariosExample feature file.

15 Scenarios ([32m15 passed[0m)
15 Steps ([32m15 passed[0m)
0m6.290s

PASSED: runScenario(“Number Three”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Four”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Five”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Two”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number One”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Multiple Rows Parallel”, “Outlines Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Seven”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Six”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Nine”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Eight”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios
PASSED: runScenario(“Number Ten”, “ScenariosExample Example Feature File”)
        Runs Cucumber Scenarios

===============================================     

Default test 
Tests run: 15, Failures: 0, Skips: 0

=============================================== ===============================================

Default suite
Total tests run: 15, Failures: 0, Skips: 0

===============================================

Code Reference

Code for this article can be found in the GitHub Repository: https://github.com/grajk88/cucumber-scenarios-parallel-example

Conclusion

This is how we achieve parallelism with Cucumber Scenarios which are atomic and have no dependency on each other. The main advantage here is the time reduction as the test execution happens from scenario-level and not from the feature-file level.