Parameterized & Data-Driven Distributed Tests using OpenSDK

In this article, I will walk you through a new feature that was introduced in the recent OpenSDK, that allows you to migrate parameterized JUnit 5 tests into TestProject 💡

What’s in it for me?

TestProject is a free test automation platform for web, mobile, and API testing. With TestProject OpenSDK, users save a bunch of time and enjoy the following benefits out of the box:

• 5-minute simple Selenium and Appium setup with a single Agent deployment
• Automatic test reports in HTML/PDF format (including screenshots)
• Collaborative reporting dashboards with execution history and RESTful API support
• Automatic distribution and deployment of test artifacts in case uploaded to the platform
• Always up-to-date with the latest and stable Selenium driver version
• A simplified, familiar syntax for both web and mobile applications
• Complete test runner capabilities for both local and remote executions, anywhere
• Cross-platform support for Mac, Windows, Linux, and Docker
• Ability to store and execute tests locally on any source control tool, such as Git

Table of Contents

• Simple Vanilla Selenium & Junit 5 Test
• The Same Test with Parameters 
• Convert to OpenSDK – We will see how we can migrate the parameterized test to OpenSDK
• Package – We will see how to build the test package in order to upload it to the TestProject Platform
• Upload – We will upload the package to TestProject
• Execute – We will execute the package from the platform
• Execution Test Reports
• Summary

---

Simple Vanilla Selenium & Junit 5 Test

Let’s start with an example of a simple, Vanilla Selenium & Junit5 test. Assuming that we have the following simple project that contains: 

1. SimpleTest.java:

import org.junit.jupiter.api.*;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class SimpleTest {

   private ChromeDriver driver;

   @BeforeEach
   public void setup() {
       driver = new ChromeDriver();
   }

   @AfterEach
   public void teardown() {
       driver.quit();
   }

   @Test
   public void loginTest() {

       // Navigate to TestProject Example website
       driver.navigate().to("https://example.testproject.io/web/");

       // Login using provided credentials
       driver.findElement(By.cssSelector("#name")).sendKeys("User");
       driver.findElement(By.cssSelector("#password")).sendKeys("12345");
       driver.findElement(By.cssSelector("#login")).click();

       // Validate successful login
       WebElement logoutButton = driver.findElement(By.cssSelector("#logout"));
       Assertions.assertTrue(logoutButton.isDisplayed(), "Login failed! ");

   }
}

The following test is a simple login test that performs a login on our demo website.

2. build.gradle:

plugins {
   id 'java'
}

group 'io.testproject'
version '1.0-SNAPSHOT'

repositories {
   mavenCentral()
}

dependencies {

   // Selenium
   implementation group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '3.141.59' // Removed

   // JUnit5
   testImplementation 'org.junit.jupiter:junit-jupiter-api:5.5.1'

}

We may want to adjust this test to accept parameters in case we want to test multiple login credentials. I will now show how to add parameters to the test so you will be able to run it with multiple values.

The Same Test with Parameters

We will now make the simple test a parameterized test. In the build.gradle file, we need to add the following library in the dependencies block:

// JUnit5 - For @ParameterizedTest
testImplementation group: 'org.junit.jupiter', name: 'junit-jupiter-params', version: '5.7.2'

You should put this declaration under the dependencies block. To change the test to @ParameterizedTest test, we need to: 

1. Replace @Test annotation with @ParameterizedTest annotation provided by the JUnit 5 framework.
2. Add parameters to the loginTest() method. In this example, we will only add a password parameter.
3. Add the parameters source. In this example, we will use the @ValueSource annotation.

📌 To use other sources such as @MethodSource, @CsvSource, etc, please refer to JUnit 5 documentation.

After our changes, the new code will be: 

@ParameterizedTest // Instead of @Test annotation
@ValueSource(strings = {"12345", "ABCDE"}) // Added for providing parameters to password argument
public void loginTest(String password) { // Instead of: loginTest()

   // Navigate to TestProject Example website
   driver.navigate().to("https://example.testproject.io/web/");

   // Login using provided credentials
   driver.findElement(By.cssSelector("#name")).sendKeys("User");
   driver.findElement(By.cssSelector("#password")).sendKeys(password);
   driver.findElement(By.cssSelector("#login")).click();

   // Validate successful login
   WebElement logoutButton = driver.findElement(By.cssSelector("#logout"));
   Assertions.assertTrue(logoutButton.isDisplayed(), "Login failed! ");

}

Executing this test will produce the following output:

Convert to OpenSDK

Converting the parameterized test to use OpenSDK is a very simple task! All we need is to follow the following instructions:

1. Replacing native Selenium chrome driver with TestProject driver

Replace this line in build.gradle: 

// Selenium
implementation group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '3.141.59'

With:

// TestProject OpenSDK
implementation 'io.testproject:java-sdk:1.2.3-RELEASE'

2. Setting compatibility mode to JDK 11 (To be able to upload to TestProject Platform)

The next thing we need to add is:

sourceCompatibility = 1.11
targetCompatibility = 1.11

This is required later when we will package and upload the test to TestProject. We need to tell Gradle to build the project with compatibility mode for JDK 11. Otherwise, we will not be able to upload the test to the TestProject platform.

3. Add -parameters options to compileTestJava Gradle task

Add the following line to build.gradle:

compileTestJava.options.compilerArgs.add '-parameters'

This is required in order to preserve the method argument names in the final jar. We need it so that when we upload the code, TestProject will be able to get the names of the parameters of the test.

4. Modify the jar task to include the tests in the final jar file

When uploading a jar file (test bundle) to TestProject, it is important to include the tests inside the jar file. To do this, add the following modification for the jar task:

jar {
   // Include compiled test classes and their sources
   from sourceSets.test.output+sourceSets.test.allSource

   // Collect and zip all classes from both test and runtime configurations
   from { configurations.testRuntimeClasspath.collect { it.isDirectory() ? it : zipTree(it) } }
}

After you made the changes above, please make sure that Gradle loaded it.

After our changes, the final build.gradle will be:

plugins {
   id 'java'
}

group 'io.testproject'
version '1.0-SNAPSHOT'

repositories {
   mavenCentral()
}

sourceCompatibility = 1.11 // Added
targetCompatibility = 1.11 // Added

compileTestJava.options.compilerArgs.add '-parameters' // Added

dependencies {

   // TestProject OpenSDK
   implementation 'io.testproject:java-sdk:1.2.3-RELEASE' // Replaced with TestProject OpenSDK

   // JUnit5
   testImplementation 'org.junit.jupiter:junit-jupiter-api:5.5.1'

   // JUnit5 - For @ParameterizedTest
   testImplementation group: 'org.junit.jupiter', name: 'junit-jupiter-params', version: '5.7.2' // Added

}


// We added the following jar task modification
jar {
   // Include compiled test classes and their sources
   from sourceSets.test.output+sourceSets.test.allSource

   // Collect and zip all classes from both test and runtime configurations
   from { configurations.testRuntimeClasspath.collect { it.isDirectory() ? it : zipTree(it) } }
}

In the SimpleTest.java, we need to:

1. Replacing import statements of Selenium driver to use TestProject driver

After we told Gradle to download the OpenSDK, we will need to tell the code to use it. First, we will need to import ChromeDriver from TestProject instead of Selenium.  For this, replace:

import org.openqa.selenium.chrome.ChromeDriver;

With:

import io.testproject.sdk.drivers.web.ChromeDriver;

2. Replace any assertion with explicit reporting

Regular assertions such as assertEquals, assertTrue, etc will not be reported by default. We should use explicit reporting instead. In our case, we need to replace:

Assertions.assertTrue(logoutButton.isDisplayed(), "Login failed!");

With:

driver.report().step("Check successful login",logoutButton.isDisplayed());

For more information about driver.report(), Read more here. 

3. Setting up Token for TestProject driver

TestProject driver requires you to set a dev token if you plan to use it locally. To set up a token, please set the environment variable TP_DEV_TOKEN with token value. You can get the token from the sdk/integrations page 📃 For more information, please refer to this page. 

After our changes, the new code with test parametrization will be:

//import org.openqa.selenium.chrome.ChromeDriver; // Removed
import io.testproject.sdk.drivers.web.ChromeDriver;

public class SimpleTest {

   private ChromeDriver driver;

   @BeforeEach
   public void setup() throws InvalidTokenException, AgentConnectException, ObsoleteVersionException, IOException {
       driver = new ChromeDriver();
   }

   @AfterEach
   public void teardown() {
       driver.quit();
   }

   @ParameterizedTest
   @ValueSource(strings = {"12345", "ABCDE"})
   public void loginTest(String password) { // Instead of: loginTest()

       // Navigate to TestProject Example website
       driver.navigate().to("https://example.testproject.io/web/");

       // Login using provided credentials
       driver.findElement(By.cssSelector("#name")).sendKeys("User");
       driver.findElement(By.cssSelector("#password")).sendKeys(password);
       driver.findElement(By.cssSelector("#login")).click();

       // Validate successful login
       WebElement logoutButton = driver.findElement(By.cssSelector("#logout"));

       // Assertions.assertTrue(logoutButton.isDisplayed(), "Login failed!"); // Removed
       driver.report().step("Check successful login",logoutButton.isDisplayed()); // Added

   }
}

That’s it! 

If you don’t want to package and upload the test, you can stop here. You can run the test as it is now and the test report will be uploaded to the TestProject Platform automatically by the TestProject driver 😉 

If you do want to upload the test into your TestProject account to benefit from full team collaboration and built-in CI flows, keep reading 👇

Package

We will now see how to build the test package in order to upload it to the TestProject platform. To package the test, we need to use TestProjectParameterizer.class as the argument source.

Replace:

@ValueSource(strings = {"12345","ABCDE"})

With:

@ArgumentsSource(TestProjectParameterizer.class)

That’s it! Now all that is left is to run the Gradle jar task to generate the test package!

Upload

The next thing we will do is upload the package. To upload the jar file, follow these steps:

1. Click on button within your TestProject account: 
2. Select the “Code” option, click “Next”:
    
3. Follow the wizard steps

Once the test is uploaded, you should see it here, in your project area:

Execute

The final thing we need to do is to execute the package from the platform. There are 2 ways to execute the parameterized test:

1. Specify the parameters manually
2. Use a CSV file

1. Specify the parameters manually 

When you execute the test, you will have the option to specify the parameters of loginTest(String password) via the wizard:

2. Test parametrization using CSV file (data-driven execution)

In addition, there is also an option to select a CSV file for the test and add the test to the job. To add a CSV file, first, we will generate a CSV template and fill the template with values. Finally, we can configure the CSV file for the test. Follow the steps below:

1. Download CSV template by clicking on the following button:
   
2. Open the template in CSV editor (like excel or even notepad) and fill in the values:
   
   
   (In this case, I added rows 2 and 3)
3. Upload it to TestProject 
4. Use it and assign the CSV to the test
5. Execute the test (this time with the CSV)

For more information about how to upload CSV to TestProject and how to use it, you can read the documentation.

View the execution report

This time we got the following report:

As you can see, now we have 2 tests here:

Each test is a different iteration and each iteration comes from the CSV. As you can see, the first iteration passed and the second iteration failed. This is because in the second iteration we used the wrong password so the login failed ❌ If we open the steps report of the second iteration, we can see what actually failed:

What have we learned so far? 

• Convert JUnit 5 test to parameterized tests
• Migrate the parameterized test to be supported by TestProject
• Package the parameterized test, upload it and execute it
• How we can run the packaged parameterized test from the platform (by CSV or specifying the parameters manually)

There is of course more to learn about the OpenSDK, such as custom reporting instead of automatic reporting, and much more. This article just covers the basic test parametrization so you will be able to get started.

That’s it for the article! Hope the guide helped you 😊