Testing with Jenkins

You can use the Jenkins plugins for performance and Web UI testing to run tests on a Jenkins server by using a Jenkins build step.

Introduction

To automate testing with Jenkins, you must configure two computers: A Jenkins master and a Jenkins slave. This master-slave configuration allows a single Jenkins installation on the master computer to host multiple slave environments for building and running tests. You must install the newest version of the Jenkins plugins on the master computer, and install the products themselves on the slave, where you create the tests. For detailed information about the Jenkins master-slave relationship, see the Distributed Builds section on the Jenkins site.

Note: Version 2.0 of the Jenkins plugin is now available and will work with versions 9.2 and newer of the testing products. If you have version 1.0 of the Jenkins plugin, you must uninstall it before installing version 2.0.

Before you begin

  • Install IBM® Installation Manager, which is required for installing the products.
  • Install the products.
  • Verify that you have a Jenkins master computer where Jenkins and the Jenkins plugins are installed.
  • Verify that you have a Jenkins slave computer where the products are installed.
  • Verify that you have a test residing within an Eclipse workspace on the Jenkins slave where the products are installed.
Note:
  • Running Web UI tests in the Chrome browser on Linux or Mac OS requires version 9.2 or newer of the testing product. For Linux or Mac OS, you must add an environment variable that points to the installation directory of the product:, for example, export TEST_WORKBENCH_HOME=/opt/HCL/HCLOneTest. For Windows, this environment variable is already defined.
  • If you installed the product on Windows using the stand-alone installer (not using IBM Installation Manager), you must specify the environment variable that points to the installation directory of the product. For example, export TEST_WORKBENCH_HOME=C:\Program Files\HCL\HCLOneTest.

Installing the Jenkins plugins on the Jenkins master computer

  1. Download the Jenkins plugin for the appropriate test product from HCL License & Delivery Portal on to the computer where the test product is installed.
  2. From the Jenkins dashboard, install the Jenkins plugin for one or both test products.
  3. Configure Global Security to allow Random TCP Ports for JNLP (Java™ Network Launch Protocol) agents.

Job Configuration

  1. Create a new Jenkins free-style software project. For details, see https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project.
  2. From the Jenkins dashboard, click Build > Add build step and click Run a product name test.

  3. Provide details about the test run as described below.

    Build page defining test parameters

    The following table explains each field.

    Field Description
    Name Required. The name of the test.
    Workspace Required. Complete path to the Eclipse workspace.
    Project Required. The path, including the file name, of the project relative to the workspace.
    Test Suite Name Required. The path, including the file name, of the test to run relative to the project.

    To run multiple tests of the same project sequentially, specify test names separated by comma.

    Var File Optional. The complete path to the XML file that contains the variable name and value pairs.
    Config File Optional. The complete path to a file that contains the parameters for a test or schedule run.
    Results File Optional. The name of the results file. The default result file is the test or schedule name with a time stamp appended. The results file is stored in the Results directory. If you are running multiple tests, do not provide a name for the results file.
    Overwrite Results File Optional. Determines whether a results file with the same name is overwritten. The default value, false, means that the results file will not be overwritten.
    Quiet Optional. Turns off any message output from the launcher and returns to the command shell when the run or the attempt is complete.
    Number of Virtual Users Optional. HCL OneTest™ Performance only. Overrides the default number of virtual users in the run. For a schedule, the default is the number of users specified in the schedule editor. For a test, the default is one user.
    Vm Args Optional. Java virtual machine arguments to pass in.
    Exported Statistical Report Data File Optional. The complete path to a directory in which to store exported statistical report data.
    Custom Report Format File Optional. A comma-separated list of absolute paths to custom report format files (.view files) to use when exporting statistical report data with the Export Statistical Report Data File option.
    Exported Statistical Report in html Optional. The complete path to a directory in which to export web analytic results. Analyze the results on a web browser without using the test workbench. If you run multiple tests, do not provide a value in this field. The web analytic results will be exported to Jenkins workspace.
    User Comments Optional. Add text within double quotation mark to display it in the User Comments row of the report.
    -IMShared Location Complete path to HCLIMShared location, if it is not the default location.

    If you do not supply a value for Exported Statistical Report Data File, these logs will be saved in the Jenkins workspace/temp directory.

  4. Click Save to save the build step configuration.
  5. To run multiple test under the same job, click Add build step again and provide details for the next test.

Master/Slave Configuration

Master and slave configurations are supported by this plugin. See the Jenkins documentation on Distributed builds for more information.

While creating the job configuration, in addition to the steps above, you must provide the name of the slave node as the Label Expression in the Restrict where this project can be run field. This is the location where the products are installed and where tests can be run.

Restrict where this project can be run

Running tests

After creating the Job (project), open the project and click Build Now. This action starts the test run on the slave computer. You can run multiple tests of the same project by specifying test names (../<testName>) separated by comma in Test Suite Name. Specify the relative path from the project to the test including the file name of the test. To run multiple tests from different projects, add new build step for each project.

Build result and logs

  1. After the build completes, click the build number and open the console for the project. Look for Test Result to check the test execution status.
    Note: If you added multiple build steps to run multiple tests, you will find more than one Test Result.
  2. For product logs, log in to the slave machine and look in the Jenkins slave workspace/temp directory.
Feedback