Jperf

The jperf Framework
The jperf performance testing framework is part of The Fed Library. It is a performance testing tool that currently supports Portico and RTI-NG.

The basic goal of jperf is to run a federation with a number of worker federates and store performance information.

Obtaining jperf
You can download jperf from the Portico Download area, or you can get it in source code form as part of The Fed Library (see that article for more details).

Running jperf
The jperf tool has two main components: the execution manager and the worker federate. Each jperf federation can contain at most one execution manager, but can have as many worker federates as you wish.

Starting a jperf test is simple:

1. Ensure your RTI is up and running

See Getting Started with Portico for Portico-specific details.

2. Start the execution manager

From the jperf distribution directory, issue the following command:

[tim@zapp jperf-1.0]$ bin/exmanager INFO [main] jperf: Welcome to JPerf v1.0: Portico Performance Testing INFO [main] jperf: Starting EXECUTION MANAGER INFO [main] jperf:  >>>>> press return to begin the test <<<<<

At this point, the execution manager will stall until you press enter. This gives you a chance to get all your worker federates into the federation

3. Start the Worker Federates

Start however many worker federates you want:

[tim@zapp jperf-1.0]$ bin/jperf --timestepped --objects 10 --interactions 10 --iterations 10 INFO [main] jperf: Welcome to JPerf v1.0: Portico Performance Testing INFO [main] jperf:  === configuration === INFO [main] jperf:   -> endpoint:      null (Portico specific) INFO [main] jperf:   -> rti binding:   portico INFO [main] jperf:   -> obj-count:     10 INFO [main] jperf:   -> interactions:  10 INFO [main] jperf:   -> iterations:    10 INFO [main] jperf:   -> timestepped:   true INFO [main] jperf: Initialized handles INFO [main] jperf: Published and Subscribed INFO [main] jperf: Enabled Time Constrained and Regulating INFO [main] jperf: Waiting on Synchronization Point: READY_TO_REGISTER


 * The command line arguments are outlined later in this document.

Once the worker federate has done all of its preliminary work, it will block until the point  is synchronized on. This won't happen until you hit enter over on the execution manager

4. Start the Test

Go back over to the execution manager and hit enter. At this point, the test will go off and do its own thing. It will clean itself up once the run is complete. The results for each worker federate are printed in its window when it completes. Expect to see something like this:

INFO [main] jperf: Registering 10 instances INFO [main] jperf: Registered 10 instances INFO [main] jperf: Waiting on Synchronization Point: READY_TO_RUN INFO [main] jperf: Beginning 10 iterations INFO [main] jperf: Finished 10 iterations INFO [main] jperf: Deleted 10 instances INFO [main] jperf: Resigned from federation INFO [main] jperf:  ======== Test Results ======== INFO [main] jperf:  Federate Name:     jperf:253 INFO [main] jperf:  Iteration Time:    632ms INFO [main] jperf:   >>   Sent Stats   << INFO [main] jperf:  Updates Sent:      100 INFO [main] jperf:  Interactions Sent: 100 INFO [main] jperf:   >> Callback Stats << INFO [main] jperf:  Discover Count:    0 INFO [main] jperf:  Reflect Count:     0 INFO [main] jperf:  Interaction Count: 0 INFO [main] jperf:  Delete Count:      0 INFO [main] jperf:   >>   Time Stats   << INFO [main] jperf:  Time Advances:     10 INFO [main] jperf:  Total Wait:        23ms INFO [main] jperf:  Average Wait:      2.3ms

In this case, there was only one worker federate in the test federation. A jperf worker runs for X iterations. During each iteration it will update:


 * Update all attributes of each instance it registered
 * Send X interactions
 * Request a time advance (only if it is timestepped)

The jperf federate will do this for however many iterations it is configured to run with (defaults to 10).

Understanding jperf Results
A typical results table would look like that provided in the previous subsection. The meaning of each item is given below:

Testing the Portico JVM Binding
The jperf code includes a special Portico-specific command line argument. If issues, this argument will make use of the Portico JVM Binding that runs federates as separate threads in the same execution (thus vastly reducing overhead).

To start a JVM jperf test, use the  argument (where x is the number of workers to start).

[tim@zapp jperf-1.0]$ bin/jperf --jvm 4 --timestepped

The command above will start a JVM binding jperf test with 4 worker federates (and an execution manager). The federates will also be timestepped (they will have regulating/constrained on also). The log output for each federate will print in the same window, and once you can see that all workers have started and are waiting to go, just hit enter and they'll be off.

Once the run has completed, the results for each federate will be printed out to the screen. If you are running a number of worker federates, it is probably a wise idea to redirect this output to a log file, or make sure that your terminal buffer is large enough to hold it all.

jperf Command Line Arguments and Configuration
All jperf configuration is done from the command line. Each of the configuration arguments is described below:

jperf and RTIs
The jperf tools uses an RTI abstraction layer known as the "wrapper" (currently available inside the Middlesim subversion repository). At the moment, bindings exist for Portico and DMSO's RTI-NG. See the  command line argument above for more info.