Exploring TestNG .xml File

  • In last chapter, we discussed about Single test case class of two types which can be a soft or hard assert.

  • In industry, we will never have a single class, instead we will have many classes for every functionality such as ‘loginTestCases.java’, ‘ForgotPasswordTestcases.java’ and many more. So it’s not possible to run each test class separately.
    We need to do integration between test classes and so now testng.xml comes into the picture. We need testng.xml for integrating test case classes and many other things which are explained later in this chapter.

  • To start our project we will follow the below given steps:

    • Create ‘LoginTestCases.java’ and ‘ForgotPasswordTestCases.java’.

    • Then start putting test cases inside every class.

    • Make one test case pass and the other fail in each class which means there will be total 2 test cases that will pass [one in every class].

See the below given code.

  • First right click on project >> go to ‘convert to testng’ >> next >> finish.
    Now we can see “testng.xml” generated in our project.

testng .xml file

This file will look like:testng .xml file

  • Now right click on ‘testng.xml’ >> run as >> testNG suite.

See console or go to window as shown below:testng suite

  • See test-output folder and open “index.html” in a browser.

  • In live company projects this file is created only once.

  • Every automation engineer has to place his class name in the file once test class is created so that it will start running in integration suite.

  • One more thing is that we need to explore “@Test” annotation in detail.

In the above snap priority is attribute of test annotation, we have many attributes like this for test annotation.

Attribute Names Description
alwaysRun If set to true, this test method will always be run even if it depends on a method that failed.
dataProvider The name of the data provider for this test method.
dataProviderClass The class where to look for the data provider. If not specified, the data provider will be looking at the class of the current test method or one of its base classes. If this attribute is specified, the data provider method needs to be static in the specified class.
dependsOnGroups The list of groups this method depends on.
dependsOnMethods The list of methods this method depends on.
description The description for this method.
enabled Whether methods on this class/method are enabled.
expectedExceptions The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.
groups The list of groups this class/method belongs to.
invocationCount The number of times this method should be invoked.
invocationTimeOut The maximum number of milliseconds this test should take for the cumulated time of all the invocation counts. This attribute will be ignored if invocationCount is not specified.
priority The priority for this test method. Lower priorities will be scheduled first.
successPercentage The percentage of success expected from this method.
singleThreaded If set to true, all the methods on this test class are guaranteed to run in the same thread, even if the tests are currently being run with parallel=”methods”. This attribute can only be used at the class level and it will be ignored if used at the method level. Note: this attribute used to be called sequen- tial (now deprecated).
The size of the thread pool for this method. The method will be invoked from multiple threads as specified by invocation Count.
Note: this attribute is ignored if invocation Count is not specified

Important is creating a group.

Now in testng.xml we can write about this group.

Generally people create groups like critical medium as per severity, some may create like smoke, regression etc.

Different flavors of “ testng.xml” as below.

You can specify package names instead of class names:

  • In this example, TestNG will look at all the classes in the package “test.sample” and will retain only classes that have TestNG annotations.

  • You can also specify groups and methods to be included and excluded:

  • You can also define new groups inside “testng.xml” and specify additional details in attributes, such as whether to run the tests in parallel, how many threads to use, whether you are running Junit tests, etc.

  • By default, TestNG will run your tests in the order they are found in the XMLfile. If you want, the classes and methods listed in this file to run in an unpredictable order, set the preserve-order attribute to false.

Important is creating a group.

Now run above example and try observing output, it should look like below:

parallel tests using testng

The above result shows that two methods are executed using different threads. When we run the same “testng.xml” again, the result may vary. The assigning of the thread is taken care by the processor. So we can’t say which thread is going to execute which method.

Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the test-output folder of TestNg project. This XML file contains the necessary information to rerun only these methods that failed, utilizing this we can easily rerun the failed tests without having to run the entire test suite.

For that we have to follow below steps:

Create one class with two to three TCs as below:

As seen above we are going to fail TC intentionally as expected and actual result differ.

Assert.assertEquals(“test”, “test1”)

Now Run the TestNG class and you will able to see Test-Output folder as seen below:

    Locate the testng - failed.xml file
    Project -> test-output -> testng-failed.xml

testng failed.xml file

Open the file and as above we can see that the only failed method are included in include tag in xml file. Now if we rerun this file only Failed TC will execute.