Writing Unit Tests

The Portico code base includes a suite of unit tests that help to validate the behaviour and function of the Portico RTI. Although these tests don't cover all of the Portico functionality available to this point, there has been a distinct emphasis on expanding these tests and ensuring that they are in active use during development. This article is aimed at developers and explains how to write unit tests for Portico.

Testing Overview
The Portico Unit tests can be split into two categories:

1. Unit tests based on the HLA API

Unit tests based on the HLA API comprise the major portion of tests currently implemented with Portico. These tests are collectively known as HLAUnit.

These are test classes that interact with Portico via the standard HLA interface, rather than the underlying Portico interface. At the moment, all the tests are coded to the standard HLA 1.3 interface, but an expansion into the other interfaces is planned.

The code in the package comprises the current suite of HLA API focused unit tests.

2. Unit tests focused on internal components

Unit tests that are focused on testing the internal behaviour of Portico components should appear in the same package as the class it is attempting to test (only in the  directory, not  ). For example, if there is code that is testing the class, it should be in a class called.

There isn't much of this code at the moment, as most of the focus has gone on testing through the HLA API. However, this is an area that will be expanded upon in the future.

Writing Tests for HLAUnit
The problem with writing unit tests against the HLA interface is that most of the situations require considerable setting up. That is, if you want to test that an attribute reflection is being sent properly, you first have to create and join a federation, announce your publication interests and then register an object.

Further, even after you have done all this, you need to do it again for a second federate, and you have to write a separate  implementation for that federate that will probe any reflect callback it receives for the appropriate information. Then, once all that is completed, you have to do it again with different parameters to see what the result is in certain expected success and error situations. This whole process is cumbersome and makes testing all but the most basic HLA functionality difficult.

The HLAUnit framework provides a number of helper entities that remove much of this burden, allowing the developer to only use the HLA API for the functionality he is trying to test. Using the provided components you can remove much of the infrastructure development and get down to testing what you actually want to be testing.

Basic Test Structure
There is a basic structure that all HLAUnit tests should follow. The template below shows what this is:

 package hlaunit.hla13;

import org.testng.Assert; import org.testng.annotations.AfterClass; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test;

@Test(sequential=true, groups={"groupNameOne","groupNameTwo"}) public class MyTest extends Abstract13Test {   //--    //                    STATIC VARIABLES //--

//--   //                   INSTANCE VARIABLES //--   //protected Test13Federate defaultFederate; --inherited from Abstract13Test private Test13Federate secondFederate;

//--   //                      CONSTRUCTORS //--

//--   //                    INSTANCE METHODS //--

@BeforeClass public void beforeClass {       super.beforeClass; }   @BeforeMethod public void beforeMethod {       this.secondFederate = new Test13Federate( "secondFederate", this ); defaultFederate.quickCreate; defaultFederate.quickJoin; secondFederate.quickJoin; }   @AfterMethod public void afterMethod {       defaultFederate.quickResign; secondFederate.quickResign; defaultFederate.quickDestroy; }   @AfterClass public void afterClass {       super.afterClass; }   ////////////////////////////////////////////////////////////////////////////    /////////////////////////////// Test Methods /////////////////////////////// ////////////////////////////////////////////////////////////////////////////

/////////////////////////////   // TEST: testSomeFeature // /////////////////////////////   /**     * This test is provided to test the X feature. It uses the default * federate to join, do some stuff and then die */   @Test public void testSomeFeature {       // initialize the test // defaultFederate.quickPublish( "InteractionRoot.X" );

// send an RO interaction // defaultFederate.quickSend( "InteractionRoot.X", null, null ); // request an advance // secondFederate.quickAdvanceRequest( 100.0 ); }   //--    //                     STATIC METHODS //-- }

There are a few important parts to note from this code:


 * 1) Extends Abstract13Test: All tests should extend this class. It provides a number of inherited variables that are useful to the test, and the methods to set them up and clean up once the test is done.
 * 2) The @Test Declaration: The @Test annotation is expect to be on every test class. This allows groups to be assigned to the class, which then allows tests to be selectively executed and helps cut down on development time by removing the execution of unnecessary tests.
 * 3) The Test Lifecycle Methods: There are 4 lifecycle methods for each test:,  ,   and  . Put any necessary setup/cleanup logic in these methods. It is important to note that the before and after class methods should call to the parent first so it can set up any state before the test class sets up any additional items.
 * 4) The default federate: You have access to a instance called   (this was created in the parent class)

The  class provides a number of important hlaunit specific setup functionality. Your tests should extend from it. Among other things, it provides access to the  (an instance of  ). This class provides a number of helper methods. Often, when writing a unit test, you are only interested in validating one small piece of functionality. As the HLA often requires significant setup to occur before some calls can be successfully executed, the  class allows a user to perform many commons tasks quickly.

To properly test your code, you may have to create other  instances to work with. The existing unit tests classes do this extensively. You can use this code as a template for your HLAUnit tests. Alternatively, you can browse the code for the other unit tests to see how they work.

The Testing FOM
A default FOM is used for most of the unit tests. It is a simple structure that includes some basic objects, attributes and interactions. You can find this FOM in the  file. Take a couple of minutes now to briefly look over it. It is quite a simple FOM, but it is more than adequate for testing purposes.

The Test Helper Classes
The sub-sections below introduce these classes and discuss their purpose and use. At the end, we put it all together with a real example of a live test.

Class:
All HLAUnit tests should extend the class. It has a couple of useful variables that are setup and inherited:


 * ''' : Is a reference to the
 * ''' : If a reference to the Logger that should be used for any output
 * ''' : If a reference to an instance of that will be automatically set-up for use. Often times, a federate will need to add additional  instances to complete its work. However, this is situation-dependent.

All tests are expected to follow the template provided above. If you have a particular need you can deviate from it as required, but this is strongly discouraged. Look at the existing unit test code to get an idea of how a test should operate.

Class:
The class is an abstraction of a federate. It provides a number of helper methods to perform common tasks quickly.

Each helper method expects success, and as such, will catch any exceptions and if the situation deviates from what it designates as success,  will be used to kill the tests that causes this situation. This way, test code doesn't have to handle error conditions; it can just call the appropriate helper method and expect that it succeeded.

The helper methods in focus on RTIambassador calls. Helper methods for handling callbacks are the realm of the. Each contains a instance of a  that is accessible through the   variable. The federate helper methods are generally prefixed with the string "quick" as they are meant to quickly perform a certain action.

Some examples of the kinds of helper methods provided in include:


 *   The variants of this method will create a federation
 *   The variants of this method will destroy a federation
 *   Variants join a federation
 *   Variants resign from a federation
 *   Variants publish an object or interaction class
 *   Variants register a new object instance
 *   Variants update the values of an object instance
 * more...

To see how easy it is to use these methods, consider the code snippet below. This code will:


 * 1) Create a federation
 * 2) Join the federation
 * 3) Publish some attributes of an object class
 * 4) Register an instance of that class
 * 5) Updates some of its attributes
 * 6) Reflect the update
 * 7) Resign from the federation
 * 8) Destroy the federation

 // the second parameter is the Abstract13Test the federate // is associated with Test13Federate federate = new Test13Federate( "firstFederate", this );

// 1. create a federation federate.quickCreate;

// 2. join the federation federate.quickJoin;

// 3. publish some attributes federate.quickPublish( "ObjectRoot.A", "aa", "ab", "ac" );

// 4. register an object int objectHandle = federate.quickRegister( "ObjectRoot.A" );

// 5. update some attributes & 6. send a reflection Map values = new HashMap; values.put( "aa", "blah".getBytes ); values.put( "ab", "blah".getBytes ); values.put( "ac", "blah".getBytes ); fedeate.quickReflect( objectHandle, values, null );

// 7. resign from the federation federate.quickResign;

// 8. destroy the federation federate.quickDestroy;

There are many, many helper methods provided in, the code above includes just a small example of some of them.

When testing a particular part of the HLA API, these methods are useful for getting the test into the required state. You can then invoke the appropriate interface methods. For example, even though a number of  methods exist, the reflection tests calls the HLA API directly, as its purpose is to test those methods. The helper methods are just used as a convenience to allow the a test to be constructed in a little code as possible.

Class:
Each instance contains within it a  instance. The role of the federate ambassador in this case it to collect state information as callbacks are received, and to provide methods that allow test code to query the state of that collected information. For example, the holds internally a collection of information about all discovered instances. This collection can then be queried by the test code to ensure that it is correct.

The general use of this class is based around the idea of waiting for a particular event to occur, or a condition to become true. For example, a method of the ambassador allows a test federate to wait until an object instance of a particular handle is discovered.

Any time code must wait for some event to happen, there will always be the question of exactly how long it should wait before it gives up. Internally, each method will only wait until a certain timeout value has been reached. If the expected situation has not occurred before then, an exception will be thrown and this should kill the test.

By default, the timeout value is 1 second. At some point in the future this value will be dynamically changeable. Longer values may be necessary when connections that are slower are being used instead of the JVM binding.

Some examples of these methods include:
 *   waits for constrained to be enabled
 *   waits for regulation to be enabled
 *   waits for the discovery notification of a particular object instance
 *   waits until the federation is synchronized on a particular point
 *   waits to receive an interaction of a specified type
 *   waits for a reflection of a particular object instance
 * -more...

Just as there are situations in which you want to wait for an event to occur, there are also situations in which you want to wait for an event NOT to occur (for example, you want to ensure that a particular federate that is not meant to be part of a restricted synchronization point doesn't get an announcement). The class includes methods that reverse the success definition. That is, if the event or situation DOES occur, the test will fail. If a timeout occurs instead, the test is continued happily.

Some examples of these timeout-expectant methods include:
 *   wait for an interaction that should never be received
 *   wait for a time advance that should never come
 *   wait for a reflection that should never come
 * more...

As an example of this code in action, consider the following listing. This code creates and joins two federates to a federation and coordinates a synchronization point between them before exiting:

 // the second parameter is the Abstract13Test the federate // is associated with Test13Federate federateOne = new Test13Federate( "federateOne", this ); Test13Federate federateTwo = new Test13Federate( "federateTwo", this );

// 1. create a federation federateOne.quickCreate;

// 2. join the federation federateOne.quickJoin; federateTwo.quickJoin;

// 3. announce the sync point federateOne.quickAnnounce( "SYNC_POINT", null ); federateTwo.fedamb.waitForSyncAnnounce( "SYNC_POINT" );

// 4. achieve the point and wait for it to be sync'd on federateOne.quickAchieve( "SYNC_POINT" ); federateTwo.quickAchieve( "SYNC_POINT" ); federateOne.fedamb.waitForSynchronized( "SYNC_POINT" ); federateTwo.fedamb.waitForSynchronized( "SYNC_POINT" );

// 5. resign from the federation federateOne.quickResign; federateTwo.quickResign;

// 6. destroy the federation federate.quickDestroy;

This code is a combination of the benefits provided by both the and  class. Writing the various classes and infrastructure code necessary to support this simple scenario would be difficult, if not impossible without these classes.

State Variables
The class also includes a number of accessible state variables from which information can be directly obtained (rather than using the waitXxx methods).

For example, sets of all the synchronization points that have been announced and achieved are maintained, along with groups of all the instances that have been discovered, interactions sent and the sets of attributes that were last updated.

Each of these variables is marked as  which means they are fully accessible from any class that resides inside the same pacakage. A number of tests refer to state directly. The  methods are usually used in quick setup scenarios to ensure that the proper environment is created.

Class:
For each object instance created or discovered, a separate instance of the class is created. This is a simple state maintaining class, with information about the handle, contained attributes and their values all being stored.

Instances of this class that represent discovered objects are contained within a collection inside the class in a variable called. Locally created instances are stored in a variable called  in the same class.

Class:
The class maintains state information about sent or received interactions. A list of all the received interactions is stored inside the class (in the order in which they were received). The class includes information such as the sent parameters (and their values), the class of the interaction, the tag, the timestamp and so forth.

An Annotated Example HLAUnit Test
The listing below is an extract from an actual unit test used in Portico. Most of its test methods have been removed to bring it down to a reasonable size, but it does allow you to get a feel for how all the elements of the HLAUnit framework fit together. More discussion appears below:

 /* *  Copyright 2006 The Portico Project * *  This file is part of portico. * *  portico is free software; you can redistribute it and/or modify *  it under the terms of the Common Developer and Distribution License (CDDL) *  as published by Sun Microsystems. For more information see the LICENSE file. *   *   Use of this software is strictly AT YOUR OWN RISK!!! *  If something bad happens you do not have permission to come crying to me. *  (that goes for your lawyer as well) * */ package hlaunit.hla13;

import hla.rti.SuppliedAttributes;

import org.testng.Assert; import org.testng.annotations.AfterClass; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test;

@Test(sequential=true, groups={"reflect"}) public class ReflectAttributesTest extends Abstract13Test {   //--    //                    STATIC VARIABLES //--

//--   //                   INSTANCE VARIABLES //--   private Test13Federate secondFederate; private Test13Federate thirdFederate;

private int aHandle, aaHandle, abHandle, acHandle; private int bHandle, baHandle, bbHandle, bcHandle; //--   //                      CONSTRUCTORS //--

//--   //                    INSTANCE METHODS //--   @BeforeClass public void beforeClass {       super.beforeClass; secondFederate = new Test13Federate( "secondFederate", this ); thirdFederate = new Test13Federate( "thirdFederate", this ); }   @BeforeMethod public void beforeMethod {       defaultFederate.quickCreate; defaultFederate.quickJoin; secondFederate.quickJoin; thirdFederate.quickJoin; // update the handle information // aHandle = defaultFederate.quickOCHandle( "ObjectRoot.A" ); aaHandle = defaultFederate.quickACHandle( "ObjectRoot.A", "aa" ); abHandle = defaultFederate.quickACHandle( "ObjectRoot.A", "ab" ); acHandle = defaultFederate.quickACHandle( "ObjectRoot.A", "ac" ); bHandle = defaultFederate.quickOCHandle( "ObjectRoot.A.B" ); baHandle = defaultFederate.quickACHandle( "ObjectRoot.A.B", "ba" ); bbHandle = defaultFederate.quickACHandle( "ObjectRoot.A.B", "bb" ); bcHandle = defaultFederate.quickACHandle( "ObjectRoot.A.B", "bc" ); // do publish and subscribe // defaultFederate.quickPublish( "ObjectRoot.A.B", "aa", "ab", "ba", "bb" ); secondFederate.quickSubscribe( "ObjectRoot.A.B", "aa", "ab", "ac",                                                        "ba", "bb", "bc" ); thirdFederate.quickSubscribe( "ObjectRoot.A", "aa", "ab", "ac" ); }   @AfterMethod public void afterMethod {       thirdFederate.quickResign; secondFederate.quickResign; defaultFederate.quickResign; defaultFederate.quickDestroy; }   @AfterClass public void afterClass {       super.afterClass; }   ////////////////////////////////////////////////////////////////////////////    /////////////////////////////// Test Methods /////////////////////////////// ////////////////////////////////////////////////////////////////////////////

///////////////////////////////////   // TEST: testValidReflectionRO // ///////////////////////////////////   @Test public void testValidReflectRO {       // register the instance and wait for it to be discovered // int instance = defaultFederate.quickRegister( "ObjectRoot.A.B" ); secondFederate.fedamb.waitForDiscovery( instance ); thirdFederate.fedamb.waitForDiscovery( instance ); // update all attribute values // try {           byte[] tag = "testing".getBytes; SuppliedAttributes atts = Test13Utilities.getRTIFactory.createSuppliedAttributes; atts.add( aaHandle, "aa".getBytes ); atts.add( abHandle, "ab".getBytes ); atts.add( baHandle, "ba".getBytes ); atts.add( bbHandle, "bb".getBytes ); defaultFederate.rtiamb.updateAttributeValues( instance, atts, tag ); }       catch( Exception e ) {           Assert.fail( "Unexpected exception in valid RO attribute update", e ); }       // wait for the update in the other federates // secondFederate.fedamb.waitForUpdate( instance ); Test13Instance temp = secondFederate.fedamb.getInstances.get( instance ); // ensure that it has all the appropriate values // Assert.assertEquals( bHandle, temp.getClassHandle,                            "Second federate discovered wrong class type" ); Assert.assertEquals( "aa".getBytes, temp.getAttributeValue( aaHandle ),                            "Second federate has wrong value for attribute aa" ); Assert.assertEquals( "ab".getBytes, temp.getAttributeValue( abHandle ),                            "Second federate has wrong value for attribute ab" ); Assert.assertEquals( "ba".getBytes, temp.getAttributeValue( baHandle ),                            "Second federate has wrong value for attribute ba" ); Assert.assertEquals( "bb".getBytes, temp.getAttributeValue( bbHandle ),                            "Second federate has wrong value for attribute bb" ); Assert.assertNull( temp.getAttributeValue(acHandle),                          "Second federate has wrong value for attribute ac" ); Assert.assertNull( temp.getAttributeValue(bcHandle),                          "Second federate has wrong value for attribute bc" ); thirdFederate.fedamb.waitForUpdate( instance ); temp = thirdFederate.fedamb.getInstances.get( instance ); // ensure that it has all the appropriate values // Assert.assertEquals( aHandle, temp.getClassHandle,                            "Third federate discovered wrong class type" ); Assert.assertEquals( "aa".getBytes, temp.getAttributeValue( aaHandle ),                            "Third federate has wrong value for attribute aa" ); Assert.assertEquals( "ab".getBytes, temp.getAttributeValue( abHandle ),                            "Third federate has wrong value for attribute ab" ); Assert.assertNull( temp.getAttributeValue(acHandle),                          "Third federate has wrong value for attribute ac" ); Assert.assertNull( temp.getAttributeValue(baHandle),                           "Third federate has wrong value for attribute ba" ); Assert.assertNull( temp.getAttributeValue(bbHandle),                          "Third federate has wrong value for attribute bb" ); Assert.assertNull( temp.getAttributeValue(bcHandle),                          "Third federate has wrong value for attribute bc" ); }   //--    //                     STATIC METHODS //-- }

There are a number of elements in the above listing to take note of.

The Package

Firstly, as with all HLAUnit tests, this class appears in the  package. To keep a distinction between "HLAUnit" tests and Portico unit tests, they should all be kept in this package (although this is convention more than anything else).

The Group

The  annotation at the class level declares the various groups the test class should relate to (in this case, there is only one). By using  at this level,  will also assume that all the public methods of this class (excluding those marked with other annotations) are test methods.

Further, the  property is also set in the class-level annotation. This will cause all tests contained within to run sequentially, rather than in parallel (as is the default in TestNG). This is important as the,   and   methods will use a federation whose name is based on that of the class. Thus, if more than one method is executing simultaneously (and the above methods have been used), tests will be working in the same federation and will interfere with one another.

The Variables

Although this class inherits an instance of from the parent class, more are needed to conduct these particular tests. Further, the test stores handle values in local variables so that they are accessed the same way in each test method.

 

As always, the first step is to call up to the parent class. Following this, each of the new test federate instances are created, passing the test class itself as a way to associate the instance with the test.

 

This method is executed before each test method is run. The general process is pretty simple here. The federation is created and each federate is joined. The join call causes any residual previous state to be cleared, thus preparing the federate for use in this test.

Finally, each of the handles is fetched and the publication and subscription interests of the federate are set up.

 and  

Any necessary cleanup should be done here. Each federate is resigned from the federation, and the federation is destroyed (thus taking any state with it, so further tests are not affected).

The Test Method/s

Although each public method of this class is designated as a test (as a result of using  at the class-level), we still mark each desired test method with , if for no other reason than documentation.

The specifics of the test method will depend on what exactly you are trying to test. In this case, the quickXxx methods are used to register an object instance and make sure it is discovered in each of the other federates. Following this, the actual method we are trying to test is called. You will note that the  is accessed directly and the appropriate interface method called on it (rather than using the quickXxx facilities). As this is the call we wish to test, calling it directly allows us to ensure that it is being tested in the way we want.

Finally, the test method ensures that the information is received in the other federates, validating that it is in the expected form.

Compiling and Running Tests
To run any test code, it needs to be placed in the  directory. Having written your test and placed it into the appropriate location, you can invoke it by calling:

[tim@zapp codebase]$ ./ant test

If you only want to execute the tests that belong to a particular group (or set of groups), you need to specify the  system property:

[tim@zapp codebase]$ ./ant –Dtest.groups=groupOne,groupTwo test

See Unit Tests for more information.

Writing Non-HLAUnit Tests
The HLAUnit tests are meant to interact with the HLA interface, rather than talking directly to the various Portico components themselves. However, there is only so much code that can be tested through the HLA interfaces, especially in Portico, which provides numerous extra extension and plug-in facilities to users.

Writing non-HLAUnit tests is quite similar to writing HLAUnit tests. The obvious difference is that you are not using the HLA interface. As such, the various quickXxx facilities are not available (nor are they needed). Everything related to still applies. Tests should be placed in appropriate groups, so that they can be executed in isolation, rather than as part of a whole test run.