Chapter 6. TestNG and TAP

TestNG is a Test Framework created by Cédric Beust to fill the gaps left by JUnit at that time. One of these gaps was extensibility. It was hard modify the behavior of JUnit framework, thus you had to live with what the framework provided or, literaly, hack it. One of the ways to extend TestNG is using listeners

Using listeners in TestNG you can add actions during your test execution lifecycle. In tap4j there is a Listener responsible for creating TAP Streams from your TestNG test suite. The TestNG tap4j listener generates TAP Streams for suites, groups, tests and methods. It was created based on the HTML listener. The classes used to integrate TestNG and TAP are in the package org.tap4j.ext.testng in org.tap4j:tap4j:2.0:jar.

To make your existing TestNG tests output TAP all you need to do is add tap4j listener to your project. You can do so by adding a listener to your classes or to your test suite. Although in this book we show both ways, we recommend you to use the latter.

Example 6.1. sample.MyFirstTest.java

package sample;

import org.tap4j.ext.testng.TestTAPReporter;
import org.testng.Assert;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(TestTAPReporter.class)
public class MyFirstTest
{

	@Test
	public void testCurrentTime()
	{
		Assert.assertTrue( System.currentTimeMillis() > 0 );
	}
	
}

The code above contains a sample test that asserts that the current time in millis returned by the System is greater than zero. Notice though that we added the annotation @Listeners with tap4j's Listener as its value. The TestTAPReporter listener is responsible for generating TAP Streams for tests and methods. After running your test with this listener the following files will have been generated for you: sample.MyFirstTest.tap, sample.MyFirstTest#testCurrentTime.tap. If you are using Maven these files may have been created in ${basedir}/test-output or ${basedir}/target/surefire-reports or in another directoy depending on your configuration.

Example 6.2. sample.MyFirstTest.tap

1..1
ok 1 - sample.MyFirstTest#testCurrentTime
  ---
  message: TestNG Test testCurrentTime
  severity: '~'
  source: sample.MyFirstTest#testCurrentTime
  datetime: '2011-05-16T12:47:10'
  file: sample.MyFirstTest
  line: '~'
  name: testCurrentTime
  extensions: '~'
  got: '~'
  expected: '~'
  display: '~'
  dump: '~'
  error: '~'
  backtrace: '~'
  ...

Example 6.3. sample.MyFirstTest#testCurrentTime.tap

1..1
ok 1 - sample.MyFirstTest#testCurrentTime
  ---
  message: TestNG Test testCurrentTime
  severity: '~'
  source: sample.MyFirstTest#testCurrentTime
  datetime: '2011-05-16T12:47:10'
  file: sample.MyFirstTest
  line: '~'
  name: testCurrentTime
  extensions: '~'
  got: '~'
  expected: '~'
  display: '~'
  dump: '~'
  error: '~'
  backtrace: '~'
  ...

Next we will see how to add our listeners to a TestNG suite (XML or YAML).

Example 6.4. sample.suite.xml

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
  
<suite name="Sample suite" verbose="1" >
  <listeners>
  	<listener class-name="org.tap4j.ext.testng.TestTAPReporter"></listener>
  	<listener class-name="org.tap4j.ext.testng.SuiteTAPReporter"></listener>
  </listeners>
  <test name="My first test">
    <classes>
      <class name="sample.MyFirstTest"/>
    </classes>
  </test>
</suite>

Example 6.5. sample.suite.yaml

name: Sample suite
verbose: 1

listeners: 
  - org.tap4j.ext.testng.SuiteTAPReporter
  - org.tap4j.ext.testng.TestTAPReporter

tests:
  - name: My first test
    classes:
      - sample.MyFirstTest

After executing your tests using the listeners shown above you should find the following new file in your output folder: Sample suite.tap. A new listener was added here, it is org.tap4j.ext.testng.SuiteTAPReporter. This listener is responsible for creating TAP Streams per suite and groups for you.

Example 6.6. Sample suite.tap

1..1
ok 1 - sample.MyFirstTest#testCurrentTime
  ---
  message: TestNG Test testCurrentTime
  severity: '~'
  source: sample.MyFirstTest#testCurrentTime
  datetime: '2011-05-16T12:44:08'
  file: sample.MyFirstTest
  line: '~'
  name: testCurrentTime
  extensions: '~'
  got: '~'
  expected: '~'
  display: '~'
  dump: '~'
  error: '~'
  backtrace: '~'
  ...