In this 3^rd TestNG article, we will explore Dependency Testing. Dependency Testing allows a single Test Method to depend on one or more Test Methods. This is a valuable feature when our program relies on the ordering of Test Methods.

A Test Method such as launching a web server must execute before deploying code to an environment.

Therefore, the dependent Test Method “i.e., launch” needs to Pass before executing the calling Test Method “i.e., deploy”. If launching a web server Fails then there is no server for testing the application. As a result, with Dependency Testing, our Test Report returns one Failure and several Skips (deployment and subsequent Test Methods).

Without Dependency Testing, our Test Report returns a cascade failure due to the Failed launch method. A cascade failure resembles a waterfall of failures, meaning one test method Fails then causes the remaining test to Fail. The cascade failure produces a misleading Test Report because the calling Test Method “i.e., deployment” and remaining test were not executed. By the end of this article, you will read about TestNG’s:

dependsOnMethods Attribute
dependsOnGroups Attribute
groups Attribute

Note: Code from this article is located on GitHub https://github.com/RexJonesII/TestProject-TestNG

Tutorial Chapters

The Power of Test Next Generation (Chapter 1)
Data Driven Testing (Chapter 2)
You’re here → Dependency Testing (Chapter 3)

Depends On Methods

The dependsOnMethods Attribute is an attribute of @Test Annotation. TestNG enables our program to define an array representing a Test Method name(s) that another Test Method depends on. The following is a comment screenshot via dependsOnMethods Source Code:

Let’s access New Test via TestProject to explain dependsOnMethods. We are going to use Mobile Test Type without dependsOnMethods and with dependsOnMethods. Here are the steps:

Create your free account with TestProject.
Sign in to TestProject.
Click the New Test button.
Mobile, Web, and Code are available as Test Types.

Without dependsOnMethods Attribute

Based on the above steps without a dependsOnMethods Attribute, the following code snippet has four Test Methods:

test1_OpenBrowser
test2_LogInTestProject
test3_CreateNewMobileTest
test4_LogOutTestProject

public class NoDependsOnMethods 
{
  WebDriver driver;
  
  @Test
  public void test1_OpenBrowser ()
  {
    System.setProperty("webdriver.chrome.driver", "C:\\Users\\Rex Allen Jones II\\Downloads\\Drivers\\chromedriver.exe");
    driver = new ChromeDriver ();
    driver.manage().window().maximize();		
    driver.get("https://app.testproject.io/#/home/");
  }
  
  @Test 
  public void test2_LogInTestProject ()
  {
    WebDriverWait wait = new WebDriverWait(driver, 5);
    wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("username")));
    driver.findElement(By.id("username")).sendKeys("[email protected]");		
    driver.findElement(By.id("password")).sendKeys("InvalidPassword");		
    driver.findElement(By.id("tp-sign-in")).click();
    
    String buttonNewTest = driver.findElement(By.className("div.new-test-button")).getText();
    Assert.assertTrue(buttonNewTest.equals("New Test"), "Username and/or Password Is Incorrect");
  }
  
  @Test
  public void test3_CreateNewMobileTest ()
  {		
    driver.findElement(By.className("div.new-test-button")).click();
    driver.findElement(By.xpath("//div[text()='Mobile']")).click();
  }
  
  @Test 
  public void test4_LogOutTestProject () 
  {
    Actions act = new Actions (driver);
    act.moveToElement(driver.findElement(By.className("user-default-avatar")));
    driver.findElement(By.linkText("Logout")).click();
  }
}

The code opens Chrome and attempts to log into TestProject. However, the invalid password prevents a successful log in. Therefore, the test Fails. We expect one Pass for opening Chrome, one Failure for not logging into TestProject, and two skips. The two skips should generate for creating a new web test and logging out of TestProject. However, take a look at the Test Report without a dependency:

We see one Pass and three Failures. The Test Report is misleading because test3 and test4 did not execute due to a Failed test2. That’s why TestNG introduced Dependency Testing.

With dependsOnMethods Attribute

The dependsOnMethods Attribute helps produce a Test Report based on facts. It’s a fact, our automation script did not create a new mobile test or log out of TestProject. Those Test Methods were not executed because our script never logged into TestProject. So, why were there two Failures? It failed due to no dependency. Here’s the updated program with a dependency:

public class DependsOnMethods 
{
  WebDriver driver;
  
  @Test
  public void test1_OpenBrowser ()
  {
    System.setProperty("webdriver.chrome.driver", "C:\\Users\\Rex Allen Jones II\\Downloads\\Drivers\\chromedriver.exe");
    driver = new ChromeDriver ();
    driver.manage().window().maximize();		
    driver.get("https://app.testproject.io/#/home/");
  }
  
  @Test (dependsOnMethods = "test1_OpenBrowser")
  public void test2_LogInTestProject ()
  {
    WebDriverWait wait = new WebDriverWait(driver, 5);
    wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("username")));
    driver.findElement(By.id("username")).sendKeys("[email protected]");		
    driver.findElement(By.id("password")).sendKeys("InvalidPassword");		
    driver.findElement(By.id("tp-sign-in")).click();
    
    String buttonNewTest = driver.findElement(By.className("div.new-test-button")).getText();
    Assert.assertTrue(buttonNewTest.equals("New Test"), "Username and/or Password Is Incorrect");
  }
  
  @Test (dependsOnMethods = "test2_LogInTestProject")
  public void test3_CreateNewWebTest ()
  {		
    driver.findElement(By.className("div.new-test-button")).click();
    driver.findElement(By.xpath("//div[text()='Web']")).click();
  }
  
  @Test (dependsOnMethods = { "test1_OpenBrowser", "test2_LogInTestProject" } )
  public void test4_LogOutTestProject () 
  {
    Actions act = new Actions (driver);
    act.moveToElement(driver.findElement(By.className("user-default-avatar")));
    driver.findElement(By.linkText("Logout")).click();
  }
}

Notice, the three @Test Annotations have dependsOnMethods Attribute equal to a String. The String is an array of method(s) depended on by a Test Method:

test2_LogInTestProject depends on test1_OpenBrowser
test3_CreateNewWebTest depends on test2_LogInTestProject
test4_LogOutTestProject depends on test1_OpenBrowser and test2_LogInTestProject

Note: test4 can depend on test2 without test1. However, I wanted to show how TestNG allows a method to depend on multiple test methods.

The calling Test Method will not run if the method it depends on Fails or Skips. That’s the benefit of implementing a dependency via TestNG. Let’s look at the Test Report with a dependency:

As expected, the Test Report shows one Pass, one Failure, and two skips. The dependsOnMethods Attribute is great for an uncomplicated test. We can use the dependsOnGroups Attribute for a robust test.

Groups Attribute

The dependsOnGroups Attribute utilizes the groups Attribute for Dependency Testing. We collect multiple Test Methods then provide a name for the collection. The name will be used as a String array for @Test Annotation.

TestNG allows us to place the @Test Annotation at the method level or class level. However, only the public methods are collected for the group if the @Test Annotation is placed at the class level. Here’s an example of the groups Attribute added to the top of a class:

@Test (groups = "deploy")
public class Groups_Deployment
{
  
  public void test1_DeployToDevEnvironment ()
  {
    System.out.println("1. Deploy To Dev Environment");
  }
  
  public void test2_DeployToQA1Environment ()
  {
    System.out.println("2. Deploy To QA1 Environment");
  }
  
  public void test3_DeployToQA2Environment ()
  {
    System.out.println("3. Deploy To QA2 Environment");
  }	
  
  private void test4_DeployToPrivateEnvironment ()
  {
    System.out.println("4. Deploy To Private Environment");
  }
}

Notice, there are four methods but three methods contain a public access modifier. Therefore test1, test2, and test3 are part of a group called deploy. The fourth method contains a private access modifier and does not belong to the group. One Test Method can belong to multiple groups. The following code snippet has 3 methods belonging to multiple groups at the method level using TestProject’s Platform as an example:

public class Groups_Methods
{
  @Test (groups = "initialize")
  public void test1_LaunchWebServer ()
  {
    System.out.println("1. Launch Web Server");
  }
  
  @Test (groups = "deploy")
  public void test2_DeployToEnvironments ()
  {
    System.out.println("2. Deploy To DEV, QA1, QA2 Environments");
  }
  
  @Test (groups = "unit")
  public void test3_CreateAutomatedMobileTest ()
  {
    System.out.println("3. Create Automated Mobile Test / Group = unit" + "\n");
  }
  
  @Test (groups = { "integration", "regression" } )
  public void test4_CreateAutomatedWebTest ()
  {
    System.out.println("4. Create Automated Web Test / Group = integration, regression" + "\n");
  }
  
  @Test (groups = { "regression", "acceptance" , "bug.fix"} )
  public void test5_ExtendRecordedTest ()
  {
    System.out.println("5. Extend Recorded Test / Group = regression, acceptance, bug.fix" + "\n");
  }
  
  @Test (groups = { "acceptance", "bug.in-progress" } )
  public void test6_CreateTestUsingLibraries ()
  {
    System.out.println("6. Create Test Using Libraries / Group = acceptance, bug.in-progress" + "\n");
  }
  
  @Test 
  public void test7_ExploreAnalytics ()
  {
    System.out.println("7. Explore Analytics / No Group" + "\n");
  }
}

The code snippet showed test4, test5, and test6 belonging to multiple groups at the method level. test5 and test6 have groups that contain regular pattern expressions (bug.). Did you see how test7 did not belong to a group? We are permitted to run each Test Method not belonging to a group or belonging to a group dynamically using the command line, xml file, and other manners. The main way is to use xml to run groups. We can include and/or exclude groups. The following Test Result screenshots and xml code snippet include deploy group added at the class level:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite Deployment Groups">
  <test thread-count="5" name="Test Deployment Groups">
    <groups>
    	<run>
    		<include name ="deploy"/>
    	</run>
    </groups>
    <classes>
      <class name="testproject.blog3.Groups_Deployment"/>
    </classes>
  </test> <!-- Test Deployment Groups -->
</suite> <!-- Suite Deployment Groups -->

In this example, the code snippet list groups that should be included at runtime. We can also exclude groups similarly to including a group. There’s another tag called define that can be placed within the groups tag. The define tag represents new customized groups consisting of existing groups. It’s common to add an exclude tag in the same xml with an include tag. Here’s a breakdown that shows different combinations of include and/or exclude via @Test Annotation groups at the method level.

Depends On Groups

The dependsOnGroups Attribute is related to the dependsOnMethods Attribute. Both attributes operate comparable to each other. The difference relies on the String array. Recall the dependsOnMethods Attribute defines an array representing a Test Method name(s) that another Test Method depends on. For dependsOnGroups Attribute, we specify an array of Strings describing the group our Test Method depends on. The following is a comment screenshot via dependsOnGroups Source Code:

All Test Methods that depend on a group should execute or not execute according to the dependent Group. If a group Fail then the calling Test Method will be SKIPPED. Here’s a code snippet that shows a deployment to QA1 failing while being part of a deploy group:

public class DependsOnGroups 
{
  @Test (groups = "initialize")
  public void launchWebServer ()
  {
    System.out.println("Launch Web Server");
  }
  
  @Test (dependsOnGroups = "initialize", groups = "deploy")
  public void deployQA1Environment ()
  {
    Assert.assertFalse(true);
    System.out.println("Deploy To QA1 Environment");
  }
  
  @Test (dependsOnGroups = "deploy")
  public void test1 ()
  {
    System.out.println("1st Test");
  }
  
  @Test (dependsOnGroups = "deploy")
  public void test2 ()
  {
    System.out.println("2nd Test");
  }
  
  @Test (dependsOnGroups = "deploy")
  public void test3 ()
  {
    System.out.println("3rd Test");
  }
}

All of the Test Methods (test1, test2, test3) depends on a deploy to the QA1 Environment. Therefore, the deployment must PASS before each Test Method can execute. As a result, all Test Methods are SKIPPED because the deployment FAILED due to Assert.assertFalse(true). Assert Statements are covered in the 1^st TestNG article. The following Test Results display one Pass, one Failure, and three Skips.

Can you imagine our Test Report if there were 15 or more Tests depending on a method/group? The Test Report reviewer would be overwhelmed with fear, panic, confusion, etc. 😲 That’s the beauty of TestNG’s Dependency Testing! Therefore, we have no worries with a misleading Test Report.

I invite you to read my previous TestNG articles and wish you all Happy Testing! 😎

1st TestNG Article: The Power of TestNG (Test Next Generation)
2nd TestNG Article: Data Driven Testing With TestNG

Dependency Testing with TestNG