Configuring your CI for the Eyes/GitHub integration

The article Using Eyes with GitHub describes configurations that need to be set up in the Test Manager to use Eyes with GitHub. The current article describes the configuration that needs to be set up in the CI system you use with GitHub. The configuration consists of two steps:

  • Setting up the start of the build : Setup that is done by the CI as part of the build before Eyes tests are executed. This sets environment variables that are read by the Eyes SDK to obtain the information required by Eyes to execute the test.
  • Synchronizing multiple concurrent builds : If the CI runs a build using multiple concurrent instances, then it needs to notify Eyes when all of the instances have terminated.

Setting up the start of the build

Eyes needs two values in order to run the visual tests in the builds:

  • Your Eyes API Key : This is passed by setting the environment variable APPLITOOLS_API_KEY to the value of your Eyes API key. To obtain the value of your Eyes API key, see How to retrieve your API key .
  • The batch ID : In a standalone Eyes test, the result of each test execution appears in a separate batch in the Test manager. When multiple Eyes tests are executed using a supported CI, all of the test results appear together in a single batch. The batch ID is the Git Commit Id (SHA) and is passed to Eyes in the APPLITOOLS_BATCH_ID variable.

For detailed configuration instructions for your CI, click on the relevant link below:

  • In the workflow YAML file, add the following:

    Copy 
    env:
      APPLITOOLS_API_KEY: ${{ secret.APPLITOOLS_API_KEY }}
      APPLITOOLS_BATCH_ID: ${{ github.sha }}
      APPLITOOLS_SERVER_URL : "eyesapi.applitools.com"

    • The example above assumes that you have access to the GitHub secrets service and have stored your Applitools API key in the GitHub secret name APPLITOOLS_API_KEY .

      To obtain your API key, see How to obtain your API key.

    • The APPLITOOLS_BATCH_ID value is the Git Commit Id (SHA).

    • If you run the Eyes server on a private cloud or on-premise system, then change the value assigned to the environment variable APPLITOOLS_SERVER_URL to the URL of your Eyes server.

  • If your GitHub workflow includes two or more concurrent Eyes jobs, add another dependent job that waits for these jobs to complete and notifies Eyes that the batch has completed. For example, if you have three jobs called runtests1, runtests2, runtests3 , add the following job to the YAML file:

    Copy 
    batch-completion-notification:
      # this runs after all the test jobs have completed
      needs: [runtests1, runtests2, runtests3]
      runs-on: ubuntu-latest
      steps:
        - name: Update Applitools batch status
          uses: wei/curl@v1.1.1
          with:
          args: -X POST -d -H "accept:*/*" https://eyesapi.applitools.com/api/externals/github/servers/github.com/commit/${{env.APPLITOOLS_BATCH_ID}}/complete?apiKey=${{env.APPLITOOLS_API_KEY}}

    See Synchronizing multiple concurrent builds below for details.

  • In the travis.yml file, add the following line to the script section that runs before an Eyes test runs:

    Copy 
    script:
    - export APPLITOOLS_BATCH_ID=`echo ${TRAVIS_PULL_REQUEST_SHA:=$TRAVIS_COMMIT}`

  • On your project page:

    • Click on More options and then select Settings .
    • In the Environment Variables section:

      • In the Name field put APPLITOOLS_API_KEY .

      • In the Value field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.

      • Click Add .

  • In the config.yml file in the .circlec folder, add the following command under jobs/build/steps so that it runs before the Eyes test starts:

    Copy 
    jobs:
      build:
        steps:
           - run: 
               command: |
                  export APPLITOOLS_BATCH_ID=`echo $CIRCLE_SHA1`
                  run_the_test

  • In the project dashboard :

    • Click the gear icon.
    • On the menu on the left, under Build Settings , click Environment Variables .
    • Click Add Variable .
    • In the Name field put APPLITOOLS_API_KEY .
    • In the Value field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
    • Click Add Variable .

On the Project page, click Project Settings :

  • In the setup panel, click Add New Command Line , then copy the following line to the field and press enter: 
    
export APPLITOOLS_BATCH_ID=`echo $REVISION`
  • Click on Environment Variables .
    • Click Add .
    • In the Name field put APPLITOOLS_API_KEY .
    • In the Content field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
    • Tick the Encrypt Content check box.
    • Click Create Variable .
  • In the appveyor.yml file, add the following line in the init section: 
    
init:
  - ps: $env:APPLITOOLS_BATCH_ID = if ($env:APPVEYOR_PULL_REQUEST_HEAD_COMMIT) { $env:APPVEYOR_PULL_REQUEST_HEAD_COMMIT } else { $env:APPVEYOR_REPO_COMMIT }
  • On your project page:
    • Click SETTINGS .
    • On the left panel select Environment .
    • In the Environment Variables section click Add Variable .
    • In the Name field put APPLITOOLS_API_KEY .
    • In the Value field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
    • Click Save .

For full instructions on integration with Jenkins, see Updating Jenkins build status.

Navigate to the Jenkins project configuration, go to the Execute shell window and type the following: 

export APPLITOOLS_BATCH_ID=`echo ${GIT_COMMIT}`
export APPLITOOLS_API_KEY= "your api key"

How to prepare your code

The configuration described in the previous section sets up the values of the environment variables APPLITOOLS_API_KEY and APPLITOOLS_BATCH_ID to provide the API key and batch ID respectively. Recent versions of the Eyes SDK read these environment variables to obtain this information.

How to prepare your code for a commit push action

You need to add an API call to the CI script that will inform Applitools each time there is a commit push action.

Copy 
curl -L -d '' -X POST "$APPLITOOLS_SERVER_URL/api/externals/github/push?
apiKey=$APPLITOOLS_API_KEY&CommitSha=$CIRCLE_SHA1&BranchName=$GITHUB_BRANCH&
ParentBranchName=$APPLITOOLS_PARENT_BRANCH"

  • GITHUB_BRANCH represents the full GitHub branch name, including the organization, the repository, and the branch in the format my-org/my-repo/my-branch . For example in Circle CI:

    $CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/$CIRCLE_BRANCH

  • ParentBranchName is optional. The default is the main branch of the repository.

  • The APPLITOOLS_API_KEY argument must be passed in the query and cannot be passed in the body. For example:

    https://eyes.applitools.com/api/externals/github/push?apiKey=<myApiKe> .)

Synchronizing multiple concurrent builds

By setting the APPLITOOLS_BATCH_ID environment variable, as described in the previous section, you instruct Eyes to put the results of all tests in a particular CI build in the same Test manager batch. If your CI runs multiple builds concurrently , then you need to follow two more steps:

  • In the GitHub integration page of the Test manager, follow the instructions to disable automatic batch closing described in Using Eyes with GitHub . This is a one-time configuration that needs to be done by each team for each repository that has concurrent builds.

  • You need to indicate to Eyes when all of the concurrent builds have completed. You do this by sending the Eyes server an HTTP command. How you do this depends on your environment. For example, on a Linux-based system, you can do it as follows:

    Copy 
    eyesserver="eyesapi.applitools.com"  # set this to your Eyes server name.
    githubserver="your_gituhub_server"   # as entered in the Eyes/github integration screen e.g. github.com
    apikey=$APPLITOOLS_API_KEY
    batchid=$APPLITOOLS_BATCH_ID         # this was previously set to the Commit SHA    
    url="https://$eyesserver/api/externals/github/servers/$githubserver/commit/$batchid/complete?apiKey=$apiKey"
    header="Content-Type: application/json --data `{\"serverId\": \"$githubserver", \"commitSha\": \"$batchid\"}`"    
    extraInfo="-H $header  -v -L"

    curl -X POST $url $extraInfo

    Note that various values represented here as "$xxxx" are values set in other parts of the integration:

    • $eyesserver : The URL of your Eyes server - the same value that you set when defining the Eyes server when you run visual Eyes tests (or the default shown in the snippet above).
    • $githubserver : The GitHub server you entered on the Eyes/GitHub integration screen. as described in Using Eyes with GitHub .
    • $apikey : The same API key you use when running visual Eyes tests.
    • $batchid : The unique value that defines the batch. It is the value assigned to the APPLITOOLS_BATCH_ID environment variable, as described in Setting up the start of the build above.