logo logo

Dependency Testing with TestNG

main post image

In this 3rd TestNG article, we will explore Dependency Testing. Dependency Testing allows a single Test Method to depend on one or more Test Methods. This is a valuable feature when our program relies on the ordering of Test Methods.

A Test Method such as launching a web server must execute before deploying code to an environment.

Therefore, the dependent Test Method “i.e., launch” needs to Pass before executing the calling Test Method “i.e., deploy”. If launching a web server Fails then there is no server for testing the application. As a result, with Dependency Testing, our Test Report returns one Failure and several Skips (deployment and subsequent Test Methods).

Without Dependency Testing, our Test Report returns a cascade failure due to the Failed launch method. A cascade failure resembles a waterfall of failures, meaning one test method Fails then causes the remaining test to Fail. The cascade failure produces a misleading Test Report because the calling Test Method “i.e., deployment” and remaining test were not executed. By the end of this article, you will read about TestNG’s:

TestNG Methods

Note: Code from this article is located on GitHub https://github.com/RexJonesII/TestProject-TestNG

Tutorial Chapters

  1. The Power of Test Next Generation (Chapter 1)
    1. Why Test Frameworks Are Important?
    2. Core Functions of a Test Framework
    3. How To Install TestNG
    4. Configuration Annotations
    5. Test Annotation
    6. Assertions
  2. Data Driven Testing (Chapter 2)
    1. DataProvider Annotation
    2. dataProvider Attribute
    3. dataProviderClass Attribute
    4. Cross Browser Testing
  3. You’re here → Dependency Testing (Chapter 3)
    1. dependsOnMethods Attribute
    2. dependsOnGroups Attribute
    3. groups Attribute

Depends On Methods

The dependsOnMethods Attribute is an attribute of @Test Annotation. TestNG enables our program to define an array representing a Test Method name(s) that another Test Method depends on. The following is a comment screenshot via dependsOnMethods Source Code:

dependsOnMethods

Let’s access New Test via TestProject to explain dependsOnMethods. We are going to use Mobile Test Type without dependsOnMethods and with dependsOnMethods. Here are the steps:

  1. Create your free account with TestProject.
  2. Sign in to TestProject.
  3. Click the New Test button.
  4. Mobile, Web, and Code are available as Test Types.

Without dependsOnMethods Attribute

Based on the above steps without a dependsOnMethods Attribute, the following code snippet has four Test Methods:

  1. test1_OpenBrowser
  2. test2_LogInTestProject
  3. test3_CreateNewMobileTest
  4. test4_LogOutTestProject
public class NoDependsOnMethods 
{
  WebDriver driver;
  
  @Test
  public void test1_OpenBrowser ()
  {
    System.setProperty("webdriver.chrome.driver", "C:\\Users\\Rex Allen Jones II\\Downloads\\Drivers\\chromedriver.exe");
    driver = new ChromeDriver ();
    driver.manage().window().maximize();		
    driver.get("https://app.testproject.io/#/home/");
  }
  
  @Test 
  public void test2_LogInTestProject ()
  {
    WebDriverWait wait = new WebDriverWait(driver, 5);
    wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("username")));
    driver.findElement(By.id("username")).sendKeys("[email protected]");		
    driver.findElement(By.id("password")).sendKeys("InvalidPassword");		
    driver.findElement(By.id("tp-sign-in")).click();
    
    String buttonNewTest = driver.findElement(By.className("div.new-test-button")).getText();
    Assert.assertTrue(buttonNewTest.equals("New Test"), "Username and/or Password Is Incorrect");
  }
  
  @Test
  public void test3_CreateNewMobileTest ()
  {		
    driver.findElement(By.className("div.new-test-button")).click();
    driver.findElement(By.xpath("//div[text()='Mobile']")).click();
  }
  
  @Test 
  public void test4_LogOutTestProject () 
  {
    Actions act = new Actions (driver);
    act.moveToElement(driver.findElement(By.className("user-default-avatar")));
    driver.findElement(By.linkText("Logout")).click();
  }
}

The code opens Chrome and attempts to log into TestProject. However, the invalid password prevents a successful log in. Therefore, the test Fails. We expect one Pass for opening Chrome, one Failure for not logging into TestProject, and two skips. The two skips should generate for creating a new web test and logging out of TestProject. However, take a look at the Test Report without a dependency:

Test Report without a dependency TestReport

We see one Pass and three Failures. The Test Report is misleading because test3 and test4 did not execute due to a Failed test2. That’s why TestNG introduced Dependency Testing.

With dependsOnMethods Attribute

The dependsOnMethods Attribute helps produce a Test Report based on facts. It’s a fact, our automation script did not create a new mobile test or log out of TestProject. Those Test Methods were not executed because our script never logged into TestProject. So, why were there two Failures? It failed due to no dependency. Here’s the updated program with a dependency:

public class DependsOnMethods 
{
  WebDriver driver;
  
  @Test
  public void test1_OpenBrowser ()
  {
    System.setProperty("webdriver.chrome.driver", "C:\\Users\\Rex Allen Jones II\\Downloads\\Drivers\\chromedriver.exe");
    driver = new ChromeDriver ();
    driver.manage().window().maximize();		
    driver.get("https://app.testproject.io/#/home/");
  }
  
  @Test (dependsOnMethods = "test1_OpenBrowser")
  public void test2_LogInTestProject ()
  {
    WebDriverWait wait = new WebDriverWait(driver, 5);
    wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("username")));
    driver.findElement(By.id("username")).sendKeys("[email protected]");		
    driver.findElement(By.id("password")).sendKeys("InvalidPassword");		
    driver.findElement(By.id("tp-sign-in")).click();
    
    String buttonNewTest = driver.findElement(By.className("div.new-test-button")).getText();
    Assert.assertTrue(buttonNewTest.equals("New Test"), "Username and/or Password Is Incorrect");
  }
  
  @Test (dependsOnMethods = "test2_LogInTestProject")
  public void test3_CreateNewWebTest ()
  {		
    driver.findElement(By.className("div.new-test-button")).click();
    driver.findElement(By.xpath("//div[text()='Web']")).click();
  }
  
  @Test (dependsOnMethods = { "test1_OpenBrowser", "test2_LogInTestProject" } )
  public void test4_LogOutTestProject () 
  {
    Actions act = new Actions (driver);
    act.moveToElement(driver.findElement(By.className("user-default-avatar")));
    driver.findElement(By.linkText("Logout")).click();
  }
}

Notice, the three @Test Annotations have dependsOnMethods Attribute equal to a String. The String is an array of method(s) depended on by a Test Method:

  • test2_LogInTestProject depends on test1_OpenBrowser
  • test3_CreateNewWebTest depends on test2_LogInTestProject
  • test4_LogOutTestProject depends on test1_OpenBrowser and test2_LogInTestProject

Note: test4 can depend on test2 without test1. However, I wanted to show how TestNG allows a method to depend on multiple test methods.

The calling Test Method will not run if the method it depends on Fails or Skips. That’s the benefit of implementing a dependency via TestNG. Let’s look at the Test Report with a dependency:

Test Report with a dependency Test Report with a dependency

As expected, the Test Report shows one Pass, one Failure, and two skips. The dependsOnMethods Attribute is great for an uncomplicated test. We can use the dependsOnGroups Attribute for a robust test.

Groups Attribute

The dependsOnGroups Attribute utilizes the groups Attribute for Dependency Testing. We collect multiple Test Methods then provide a name for the collection. The name will be used as a String array for @Test Annotation.

TestNG allows us to place the @Test Annotation at the method level or class level. However, only the public methods are collected for the group if the @Test Annotation is placed at the class level. Here’s an example of the groups Attribute added to the top of a class:

@Test (groups = "deploy")
public class Groups_Deployment
{
  
  public void test1_DeployToDevEnvironment ()
  {
    System.out.println("1. Deploy To Dev Environment");
  }
  
  public void test2_DeployToQA1Environment ()
  {
    System.out.println("2. Deploy To QA1 Environment");
  }
  
  public void test3_DeployToQA2Environment ()
  {
    System.out.println("3. Deploy To QA2 Environment");
  }	
  
  private void test4_DeployToPrivateEnvironment ()
  {
    System.out.println("4. Deploy To Private Environment");
  }
}

Notice, there are four methods but three methods contain a public access modifier. Therefore test1, test2, and test3 are part of a group called deploy. The fourth method contains a private access modifier and does not belong to the group. One Test Method can belong to multiple groups. The following code snippet has 3 methods belonging to multiple groups at the method level using TestProject’s Platform as an example:

TestProject Platform

public class Groups_Methods
{
  @Test (groups = "initialize")
  public void test1_LaunchWebServer ()
  {
    System.out.println("1. Launch Web Server");
  }
  
  @Test (groups = "deploy")
  public void test2_DeployToEnvironments ()
  {
    System.out.println("2. Deploy To DEV, QA1, QA2 Environments");
  }
  
  @Test (groups = "unit")
  public void test3_CreateAutomatedMobileTest ()
  {
    System.out.println("3. Create Automated Mobile Test / Group = unit" + "\n");
  }
  
  @Test (groups = { "integration", "regression" } )
  public void test4_CreateAutomatedWebTest ()
  {
    System.out.println("4. Create Automated Web Test / Group = integration, regression" + "\n");
  }
  
  @Test (groups = { "regression", "acceptance" , "bug.fix"} )
  public void test5_ExtendRecordedTest ()
  {
    System.out.println("5. Extend Recorded Test / Group = regression, acceptance, bug.fix" + "\n");
  }
  
  @Test (groups = { "acceptance", "bug.in-progress" } )
  public void test6_CreateTestUsingLibraries ()
  {
    System.out.println("6. Create Test Using Libraries / Group = acceptance, bug.in-progress" + "\n");
  }
  
  @Test 
  public void test7_ExploreAnalytics ()
  {
    System.out.println("7. Explore Analytics / No Group" + "\n");
  }
}

The code snippet showed test4, test5, and test6 belonging to multiple groups at the method level. test5 and test6 have groups that contain regular pattern expressions (bug.). Did you see how test7 did not belong to a group? We are permitted to run each Test Method not belonging to a group or belonging to a group dynamically using the command line, xml file, and other manners. The main way is to use xml to run groups. We can include and/or exclude groups. The following Test Result screenshots and xml code snippet include deploy group added at the class level:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite Deployment Groups">
  <test thread-count="5" name="Test Deployment Groups">
    <groups>
    	<run>
    		<include name ="deploy"/>
    	</run>
    </groups>
    <classes>
      <class name="testproject.blog3.Groups_Deployment"/>
    </classes>
  </test> <!-- Test Deployment Groups -->
</suite> <!-- Suite Deployment Groups -->

Test Results Test Results

In this example, the code snippet list groups that should be included at runtime. We can also exclude groups similarly to including a group. There’s another tag called define that can be placed within the groups tag. The define tag represents new customized groups consisting of existing groups. It’s common to add an exclude tag in the same xml with an include tag. Here’s a breakdown that shows different combinations of include and/or exclude via @Test Annotation groups at the method level.

Different combinations of include and/or exclude via @Test Annotation groups at the method level

Depends On Groups

The dependsOnGroups Attribute is related to the dependsOnMethods Attribute. Both attributes operate comparable to each other. The difference relies on the String array. Recall the dependsOnMethods Attribute defines an array representing a Test Method name(s) that another Test Method depends on. For dependsOnGroups Attribute, we specify an array of Strings describing the group our Test Method depends on. The following is a comment screenshot via dependsOnGroups Source Code:

dependsOnGroups Source Code

All Test Methods that depend on a group should execute or not execute according to the dependent Group. If a group Fail then the calling Test Method will be SKIPPED. Here’s a code snippet that shows a deployment to QA1 failing while being part of a deploy group:

public class DependsOnGroups 
{
  @Test (groups = "initialize")
  public void launchWebServer ()
  {
    System.out.println("Launch Web Server");
  }
  
  @Test (dependsOnGroups = "initialize", groups = "deploy")
  public void deployQA1Environment ()
  {
    Assert.assertFalse(true);
    System.out.println("Deploy To QA1 Environment");
  }
  
  @Test (dependsOnGroups = "deploy")
  public void test1 ()
  {
    System.out.println("1st Test");
  }
  
  @Test (dependsOnGroups = "deploy")
  public void test2 ()
  {
    System.out.println("2nd Test");
  }
  
  @Test (dependsOnGroups = "deploy")
  public void test3 ()
  {
    System.out.println("3rd Test");
  }
}

All of the Test Methods (test1, test2, test3) depends on a deploy to the QA1 Environment. Therefore, the deployment must PASS before each Test Method can execute. As a result, all Test Methods are SKIPPED because the deployment FAILED due to Assert.assertFalse(true). Assert Statements are covered in the 1st TestNG article. The following Test Results display one Pass, one Failure, and three Skips.

Test Results Test Results

Can you imagine our Test Report if there were 15 or more Tests depending on a method/group? The Test Report reviewer would be overwhelmed with fear, panic, confusion, etc. 😲  That’s the beauty of TestNG’s Dependency Testing! Therefore, we have no worries with a misleading Test Report.

I invite you to read my previous TestNG articles and wish you all Happy Testing! 😎

1st TestNG Article: The Power of TestNG (Test Next Generation)
2nd TestNG Article: Data Driven Testing With TestNG

Rex Jones II

About the author

Rex Jones II

Rex Jones II has a passion for sharing knowledge about testing software. His background is development but enjoys testing applications.

Rex is an author, trainer, consultant, and former Board of Director for User Group: Dallas / Fort Worth Mercury User Group (DFWMUG) and member of User Group: Dallas / Fort Worth Quality Assurance Association (DFWQAA). In addition, he is a Certified Software Tester Engineer (CSTE) and has a Test Management Approach (TMap) certification.

Recently, Rex created a social network that demonstrate automation videos. In addition to the social network, he has written 6 Programming / Automation books covering VBScript the programming language for QTP/UFT, Java, Selenium WebDriver, and TestNG.

Join TestProject Community

Get full access to the world's first cloud-based, open source friendly testing community. Enjoy TestProject's end-to-end test automation Platform, Forum, Blog and Docs - All for FREE.

Join Us Now  

Leave a Reply

Join TestProject Newsletter

Join a 20K community of readers! Always stay up-to-date with all the latest test automation trends, best practice and tips shared by leading software testing community experts across the globe!

FacebookLinkedInTwitterEmail