In this tutorial of three articles, my goal is to share knowledge regarding Test Next Generation (TestNG). TestNG is a powerful Test Framework for testing Java code. A Test Framework is a pattern for writing and running Test Scripts. Thanks to xUnit, we have a family of frameworks for testing many programming languages.

TestNG is not a part of xUnit but was influenced by xUnit. JUnit, NUnit, and PyUnit are some of the frameworks within the xUnit family. JUnit is the de facto standard for Java while NUnit is the framework for C# and PyUnit is the framework for Python. However, in the beginning, there were limitations with the xUnit family. For example, JUnit 3 had the following constraints:

Could not return values
Could not have parameters
Test Method names had to start with test

As a result of the limitations, TestNG was created and added more features. In return, JUnit 4 embraced TestNG’s features then added some of those same features. Hopefully you can learn and apply a new feature whether you are using TestNG or another Test Framework. By the end of this tutorial, you will know:

Why Test Frameworks Are Important
Core Functions of a Test Framework
Configuration and Test Annotations
Hard and Soft Assertions
Data Driven Testing
Dependency Testing

Note: Code from this article is located on GitHub https://github.com/RexJonesII/TestProject-TestNG

Tutorial Chapters

You’re here → The Power of Test Next Generation (Chapter 1)
Data Driven Testing (Chapter 2)
Dependency Testing (Chapter 3)

Why Test Frameworks Are Important?

Test Frameworks are important because it facilitates the testing process. It facilitates the testing process by allowing us to write a quick test using Annotations and Assertions. In addition, we can execute a single Test Script or a collection of Test Scripts in a Test Suite.

Experienced Automation Engineers can leverage a Test Framework to design a Page Object Model and create Automation Design Frameworks. A Page Object Model is a design pattern while Automation Design Frameworks can be applied to most automation endeavors such as Selenium and UFT formerly known as QTP. Three of the most popular Automation Design Frameworks are Data Driven Frameworks, Keyword Driven Frameworks, and Hybrid Driven Frameworks.

Core Functions of a Test Framework

The following are four core functions of a Test Framework:

Create and Execute Test Scripts
Generate A Test Report
Generate Logs
Read and Write Test Data

How To Install TestNG

We can install TestNG using an IDE (Eclipse, NetBeans, IntelliJ), Build Tool (Maven, Gradle, Ant), Command Line, or Download the TestNG jars. For scalability, a Build Tool is the preferred way to install TestNG. However, Eclipse Marketplace is one of the fastest ways to install TestNG. Here are the steps using Eclipse IDE:

Click Help > Eclipse Marketplace:

2. Enter TestNG then click Go:

3. Click all checkboxes

4. Accept Terms

Annotations

Annotations support TestNG by controlling the execution flow of our program. They are written above their respective method and prefixed with an at “@” symbol. We can place an Annotation anywhere on the editor because it’s automatically connected to the method. In this section, we’ll cover the Configuration Annotations and Test Annotation.

Configuration Annotations

Configuration Annotations are like Pre-Conditions and Post-Conditions. Our test is the condition. A Pre-Condition is executed before our test and a Post-Condition is executed after our test. The following screenshot lists each Configuration Annotation:

All @Before Annotations help us configure and set up our test. For example, we can set the system property, load the browser, and load the Application Under Test (AUT) in a @Before Annotation. The @After Annotations provide a way to clean up important things like quitting the driver after our test. Here’s a code snippet for @BeforeMethod and @AfterMethod:

@BeforeMethod
public void setUp ()
{
  System.setProperty("webdriver.chrome.driver", "C:\\Users\\Rex Allen Jones II\\Downloads\\Drivers\\chromedriver.exe");
  driver = new ChromeDriver ();
  driver.manage().window().maximize();		
  driver.get("https://testproject.io/");
}

@AfterMethod
public void tearUp ()
{
  driver.quit ();
}

Test Annotation

A Test Annotation is written as @Test and identifies our method as a Test Method. We have the option of marking our methods or entire class with @Test. After executing our test, the Console and Results tab only show results for public methods. Other methods such as default or private are allowed to be annotated with @Test but will not execute as a Test Method. The following code snippet shows an example of 3 Test Methods annotated with @Test:

@Test
public void readTestProjectBlog ()
{
  driver.findElement(By.linkText("Blog")).click();
}

@Test
public void freeTestProjectSignUp ()
{
  driver.findElement(By.linkText("Free Sign Up")).click();
}

@Test
public void logIntoTestProject ()
{
  driver.findElement(By.linkText("Login")).click();
}

Execution Order

In our program, the @Before Configuration Annotation(s) always execute first. Next, is the Test Method(s) followed by the @After Configuration(s). We have the ability to add multiple Configuration Annotations and/or Test Methods. The deciding factor are Test Requirements. A Test Requirement determines which annotations we choose for our Test Script. Here’s a description of each Configuration Annotation in their hierarchy order (from highest to lowest):

@BeforeSuite/AfterSuite – execute its method before a Test Suite and after all Test Methods in a Test Suite
@BeforeTest/@AfterTest – execute its method before a Test and after all Test Methods in a Test
@BeforeClass/@AfterClass – execute its method before a Class and after all Test Methods in a Class
@BeforeMethod/@AfterMethod – execute its method before each Test Method and after each Test Method

The following is a code snippet of TestNG xml file which helps understand a Test Suite, Test, Class, and Test Method hierarchy:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Project Suite Example">
  <test name="Test Project">
    <classes>
      <class name="testproject.blog1.CreateNewTest"/>
      	<methods>
      		<include name = "readTestProjectBlog"/>
      		<include name = "freeTestProjectSignUp"/>
      		<include name = "logIntoTestProject"/>
      	</methods>
    </classes>
  </test> <!-- Test Project -->
</suite> <!-- Test Project Suite Example -->

The xml file provides a picture regarding each Configuration Annotation execution order. However, it does not provide a picture regarding the Test Method’s execution order. By default, the Test Methods execute in alphanumeric order. Therefore, the execution order runs each Test Method in the following order:

freeTestProjectSignUp
logIntoTestProject
readTestProjectBlog

TestNG identifies the methods by looking up its annotation. As a result, we are allowed to set a priority for each Test Method by implementing an attribute. The @Test Annotation has a total of 24 attributes with the priority attribute establishing a specific execution order.

Here’s an example of each @Test Annotation with a priority attribute:

@Test (priority = 1)
public void readTestProjectBlog ()
{
  driver.findElement(By.linkText("Blog")).click();
}

@Test (priority = 2)
public void freeTestProjectSignUp ()
{
  driver.findElement(By.linkText("Free Sign Up")).click();
}

@Test (priority = 3)
public void logIntoTestProject ()
{
  driver.findElement(By.linkText("Login")).click();
}

The following screenshot shows a different execution order after implementing a priority attribute:

readTestProjectBlog
freeTestProjectBlog
logIntoTestProject

Assertions

TestNG has more than 50 Assertion methods that verify whether our test Pass or Fail. All of them are included in a package called org.testng.Assert. The assertion methods are overloaded versions of the following:

assertTrue
assertEquals
assertSame
assertNotSame
assertFalse
assertNotNull

Each Assertion contains the same signature whereby the first parameter is an actual result, the second parameter is an expected result, and the third parameter is an optional String value. The String value will only be shown if the Assertion Fails.

TestNG provides two types of Assertions: Hard Assertions and Soft Assertions. Let’s take a look at the difference between both TestNG Assertions.

Hard Assertion

A Hard Assertion stops executing a Test Method soon as an Assertion Fail. We begin Hard Assertions by writing Assert then the dot (.) operator. After writing the dot operator, a list of assertion methods are visible in the editor. The following screenshot and code snippet display TestProject’s Home page after logging into their application and four Hard Assertions:

@Test
public void logIntoTestProject ()
{
  driver.findElement(By.linkText("Login")).click();
  
  Assert.assertEquals("NewTest", "New Test", "New Test Button Is Not Available");
  Assert.assertTrue(true, "Job Monitor Is Not Available");
  Assert.assertTrue(false, "Reports Are Not Available");
  Assert.assertEquals("Developers", "Developers", "Developers Are Not Available");
}

Notice, two Assertions Fail. The first Assertion encloses a different actual and expected result while the third Assertion returns false for the assertTrue method. In our Test Report, we should see two Fail Assertions. However, the Test Report displays only one assertion from the first Assertion.

Why did our Test Report return one Assertion? The Test Report returned one Assertion because Hard Asserts are designed to exit a Test Method after its first Assertion failure. As a result, the remaining Assertions are never executed to verify if they Passed or Failed. This leads to an incomplete report. That’s why TestNG introduced Soft Assertion.

Soft Assertion

A Soft Assertion continues executing the Test Method after an Assertion failure. Other Test Frameworks do not have a Soft Assertion feature but can plug in a similar feature. First, we declare the SoftAssert class then utilize the softassert object.

WebDriver driver;
SoftAssert softassert = new SoftAssert ();

@Test
public void logIntoTestProject ()
{
  driver.findElement(By.linkText("Login")).click();
  
  softassert.assertEquals("NewTest", "New Test", "New Test Button Is Not Available");
  softassert.assertTrue(true, "Job Monitor Is Not Available");
  softassert.assertTrue(false, "Reports Are Not Available");
  softassert.assertEquals("Developers", "Developers", "Developers Are Not Available");
  
  softassert.assertAll();
}

Notice, the last line contains a method called assertAll. The purpose of assertAll is to receive all AssertionErrors if there is a failure. The following screenshots show an accurate report after executing the same Test Script with SoftAssert.

Both Assertions show up because they Fail the verification step.

Mixed Assertions

Soft Assertions are beneficial for Test Methods with multiple verification steps. However, there are situations when it’s more efficient to implement a Hard Assertion. For example, we should consider a Hard Assertion if the application does not open. There isn’t a need to continue executing if the application fails to open. Therefore, it’s best to utilize Soft Assertions as well as Hard Assertions in our program.

In this first tutorial we’ve covered why Test Frameworks are important, what are the core functions of a Test Framework, talked about configuration and test annotations and learned all about hard and soft assertions. The next article in this TestNG series is about Data Driven Testing. Check it out here! 😉

The Power of TestNG (Test Next Generation)