In a previous article, techniques were presented for automatically configuring SUT performance tuning parameters and running hundreds of stress tests in a short period of time. The motivation was to have enough useful information to improve performance by more than an order of magnitude by adjusting tuning parameters, as others have done. However, performance tuning parameters are often interrelated so one needs to examine results for all permutations of settings, one change at a time. But how can reports effectively communicate cause-and-effect relationships to users with that much data? Producing dozens of separate static charts using Excel is an improvement somewhat over just giving users tabular data. But what kind of charting engine could reduce the labor of producing charts on-line and at the same time provide better navigation through different tuning parameter settings within a single chart for the reader?
In this article, a Java applet viewer addresses these needs using an open source charting class library, embedded database-like query capabilities and charting controls that includes animation of families of curves. Using HTML applet parameter tags, an author can configure the applet viewer in a web page to focus the reader's attention on results for particular test conditions along with commentary in the same web page. Additional data for many more tests, downloaded by the applet, is only a click away to answer 'what-if' questions on system configuration or modeling parameter changes. With chart animation, trends or subtle differences in individual curves within families of performance curves become more apparent as the applet viewer automatically scans through tunable parameter settings.
The prompt to download says: <Shield Icon> "This site might require the following ActiveX control: 'J2SE Runtime Environment 5.0 Update 4' from 'Sun Microsystems, Inc.'. Click here to install..."The applet uses an Open Source charting library, JFreeChart. With the JFreeChart class files, the applet's JAR file is 1.09MB in size but is available in the browser's cache as other web pages containing the applet are viewed.
Report 1 is an example of what an on-line report using the charting applet might look like.
Report: WebLogic Server Load Test Results, 9/9/05|
Study Objectives: Measure performance at different settings of Execute Queue ThreadCount and max-beans- configuration parameters.
Application: Bean-managed Banking Client/Server
Performance Measurements: TPS, Mean Response Time, Successful Tests, and Errors
Configuration Parameters studied: Server Execute Queue ThreadCount, Max-Beans-In-Free-Pool and Max-Beans-In-Cache
Initial Chart Parameter Settings
Test Results: Max-beans-in-cache and max-beans-in-free-pool set at 15 creates a bottleneck. Errors begin to occur at X1:Grinder_Threads (agent workload) = 20 (on X axis) and X2:weblogic.kernel.Default_ThreadCount >= 15 (in the animated legend). At weblogic.kernel.Default_ThreadCount = 25, there are hundreds of errors, which causes the superimposed TPS and Successful Tests curves to drop off (25 workload threads and higher).
|Change Parameter Settings||Chart 2: Chart with Input Parameter Controls Panel|
Mouse over input labels.
Click labels with other settings.
Use Shift-Click(s) to decrease
(or use right mouse button)
Use left button to increase
|Select Multiple Statistics, Animate, Full Chart View||Chart 3: Using Statistics and Input Parameter Charting Controls Panes|
Click/drag-down horizontal controls split pane separator.
Click labels, Mean Time, Successful Tests, Errors in the Statistics Pane.
Check Animate checkbox
Click left arrow on vertical split pane separator.
Listing 1. Applet Viewer Parameter Tags
These are the parameter tags for the configuration of the applet in Chart 4.
<OBJECT type="application/x-java-applet" WIDTH=950 HEIGHT=400> <param name="CODE" value="net.viewer.GraphApplet"> <param name="archive" value="../viewer.jar"> <param name="scriptable" value="false"> <param name="initValsRunNum" value="41"> <param name="modeName" value="X1"> <param name="x1Name" value="Grinder_Threads"> <param name ="x2Name" value="max-beans-in-cache"> <param name="autoScale" value="On"> <param name="animate" value="Off"> <param name="fullChartView" value="false"> <param name ="y2Name" value="Grinder TPS,Grinder Mean time"> </OBJECT>
Chart 4: TPS and Mean Time vs. Client Workload and Max-Beans Configuration Settings
The following provides details of some of the applet parameters including the ones used for the Viewer in Chart 4.
|name ="initValsRunNum" value="41"||The initValsRunNum applet parameter configures the Viewer to use a particular testing/modeling run number for setting all non-primary (and non-secondary) input parameters to constant values. The applet in this example is configured to use "41" as its initial (Run ID) variable for charting. Run ID numbers are defined during the load testing or model simulation phase and are saved with the input parameter and testing/modeling statistics results in unique records for each run. For input parameters, each record has its Run ID and as many additional fields as there are input parameters defined in the load test/simulation. For statistics results, an associated record has this Run ID number and as many fields as there are statistics defined in the test/model. As the user changes the Viewer's controls and label selections, the constant settings for non-primary parameters will change and the value of the internal variable initially set to initValsRunNum will change accordingly.|
|name ="x1Name" value="Grinder_Threads"||The x1Name applet parameter tells the Viewer which input parameter to use as its primary independent variable. The applet in this example is configured to use "Grinder_Threads" as its primary X axis (X1) variable. The Viewer computes the set of values, which the X1 variable can take on for charting purposes by comparing all input parameter records against the initValsRunNum record. The set of X1 values used for a chart includes X1 values taken from all Run ID's where all other input parameter values equal those in the initValsRunNum record.|
|name ="x2Name" value="max-beans-in-cache"||The x2Name applet parameter tells the Viewer which input parameter to use as the secondary independent variable. The applet in this example is configured to use "max-beans-in-cache" as its secondary X axis (X2) variable. Selecting a secondary independent variable can result in multiple curves of the same color but with different symbols being substituted as the datapoint indicators on the curves as in the example above.|
|name ="y2Name" value="Grinder TPS,Grinder Mean time"||The y2Name applet parameter tells the Viewer which statistics to chart. The applet can be configured to use a comma-separated list of statistics, which in this example consists of the "Grinder TPS" and "Grinder Mean time" statistics.|
|name ="autoScale" value="On"||The autoScale applet parameter configures the initial setting of the AutoScale checkbox. In this example AutoScale is set to On. The analyst may want to instruct the reader to scan through certain non-primary input parameter values to see their effect on the performance curves. The web page author can initialize AutoScale to OFF so that as the reader clicks Input Parameter labels to cause their values to increase/decrease, the curves can be seen to move accordingly. Scaling will stay the same. With Autoscale ON, the cause and effect sensation is visually less noticeable because the curves stay centered in the chart while the scaling is adjusted.|
|name ="animate" value="Off"||The animate applet parameter configures the initial setting of the
Animate checkbox. In this example Animate is set to Off.
With Animate turned on, AutoScale is disabled so that curves move up and down in
the chart frame with fixed scaling. Only the symbol used as datapoint indicator
for the curve will appear in the X2 legend box next to the X2 value used to
produce the curve being drawn. There is a one second delay between drawing the
curves and a two second delay at the end of the sequence of curves in the family
of curves (X2 selected).
Animate is useful when curves within a family of curves are close together and it's difficult to see minor differences in the curves.
|name ="fullChartView" value="false"||The fullChartView applet parameter tells the Viewer to use the entire Viewer width to present the chart, or reserve space to show the control panels as well.|
Uses JFreeChart Open Source Charting Library