Evaluating response time results

When you initiate a test run from the test workbench, the Mobile and Web UI Statistical report displays graphs in the Step Performance view by default. This graph shows the average response time for each test step. However, there is an option which, when enabled, will collect data on the response time within the application and outside the application (in the network and server) for each selected step. With this additional data, you can identify the contribution of the application against the contribution of the network and server to an unexpected response time or breakdown for a test step.

About this task

To collect response time contribution data, the test must be started from the test workbench, command line, Rational® Quality Manager, or HCL OneTest™ Performance. When you run a test from Rational Quality Manager or HCL OneTest Performance, ensure that it contains the e2e.collect="true" variable.

Procedure

  1. Click Run Test in the test workbench.
  2. In the Run Configuration wizard, in the Performance Measurement page, click Next, and select Collect Response Time Contribution Data. When the test is completed, the Mobile and Web UI Statistical Report displays graphs that represent the response time for each step in the test. By default, only the 10 highest response times are collected during playback, but the filter can be changed as follows:
    1. Right-click anywhere on the Step performance report, and select Modify Filters in the context menu. In Modify Filters, select Count Filter: 10 highest (see figure 1) and click Edit.
    2. In the Performance Counter Filter wizard, select Filter by count. You can enter a new value in Number to Display, and select Show highest values or Show lowest values. For example, if you enter 15, the report will show the 15 highest response times or the 15 response times with the lowest values.

    Figure 1: Modify Filters

    Select Count Filter: 10 highest to modify the counter filter.
  3. Right-click any response time graph and, from the pop-up menu, select Display Step Response Time Contributions (see figure 2). A graph displays the time spent in the application On App Time, and another graph displays the time spent outside the application (in the network and server) Off App Time (see figure 3). If you do not select Collect Response Time Contribution Data when you run the test, no response time contribution data is collected for the steps in the test, and the Display Step Response Time Contributions option is unavailable. Response time is also measured for the first step, that is, the start step.
    Restriction: Off App Time is not measured for the start step of Web and Hybrid applications, for Android versions below 4.0.

    Figure 2: Response time contribution bar graphs for the 10 highest steps of the test

    The highest Step Response time for a test run

    Figure 3: On App and Off App Time for one selected step

    Response time contribution data for a step on app and off app
  4. Click the Step Performance link at the top of the Step Performance tab to return to the main page.
Feedback