ryanjames.io

Running Unity Tests with Jenkins

Running Unity Tests with Jenkins

—

by

ryanjames
in ,

Using Jenkins for the unity project is a great option to automate builds, run tests, gather reports and in my case I also use SonarQube to analyse my static code. I set jenkins up to do this previously, and then forgot how to do it so this article will hopefully benefit my future self when I inevitably forget again.

My initial goal is to create a pipeline that will perform two tasks

  1. Run unity non-playmode tests
  2. Peform static code analysis with SonarQube

Environment and Setup

I am running Unity and Jenkins both on a local Windows 11 Home machine. This did give me some difficulty setting jenkins up the “correct” way, by having a dedicated user that can log on as a service. The reason this was challenging is that windows home edition does not have Local Security Policy available by default, unfortunately I created this user a while ago and cannot remember how I did it. Jenkins has some good steps on this though https://www.jenkins.io/doc/book/installing/windows/

I am also on a user account with administrator privileges while there is a dedicated jenkins user running jenkins as a service.

Here is some other environment info

  • Java 17
  • Unity 2023.2.3f1
  • Windows 11 Home

Plugins

Creating the pipeline

The first goal is to create a pipeline that will run the unity build

We are going to create a pipeline using the declarative syntax – https://www.jenkins.io/doc/book/pipeline/syntax/

  1. Go to Jenkins home, in my case http://localhost:9080/
  2. Select “+ New Item“
  3. Give the pipeline a descriptive name
  4. Select Pipeline
  5. Customize the pipeline with triggers, descriptions, build options etc
  6. Add the script into the Pipeline > Script box, alternatively you can also use the dropdown to select Pipeline script from SCM and point it to your git repo, allowing you to version control your Jenkinsfile
  7. Select Save

We have created our pipeline, and below is my Jenkinsfile for reference

pipeline {
  agent any

  environment {
    PROJECT_NAME                = 'Emu'
    UNITY_EDITOR_PATH           = 'C:\\Program Files\\Unity\\Hub\\Editor\\2023.2.3f1\\Editor\\Unity.exe'
    UNITY_PROJECT_PATH          = 'C:\\Users\\ryanm\\Endless Book'
    UNITY_UNIT_TEST_RESULTS_FILE = 'UnityTestResults.xml'
  }

  tools {
    jdk     'JDK17'
    msbuild 'ms22'
  }

  stages {
    stage('Activate Unity License') {
      steps {
        withCredentials([
          usernamePassword(
            credentialsId: 'unity-login',
            usernameVariable: 'UNITY_USERNAME',
            passwordVariable: 'UNITY_PASSWORD'
          )
        ]) {
          bat """
            \"${env.UNITY_EDITOR_PATH}\" -batchmode -nographics -quit \
              -username %UNITY_USERNAME% \
              -password %UNITY_PASSWORD%
          """
        }
      }
    }

    stage('Run Unity Non-Playmode Tests') {
      steps {
        script {
          def runTestsCommand = "\"${env.UNITY_EDITOR_PATH}\" -batchmode -nographics " +
                                "-logFile \"${env.WORKSPACE}\\unityTestLog.txt\" " +
                                "-projectPath \"${env.UNITY_PROJECT_PATH}\" " +
                                "-runTests -testPlatform editmode " +
                                "-testResults \"${env.WORKSPACE}\\${env.UNITY_UNIT_TEST_RESULTS_FILE}\""
          bat runTestsCommand
        }
      }
      post {
        always {
          script {
            if (fileExists("${env.WORKSPACE}\\${env.UNITY_UNIT_TEST_RESULTS_FILE}")) {
              nunit testResultsPattern: "${env.UNITY_UNIT_TEST_RESULTS_FILE}"
            }
            archiveArtifacts artifacts: "${env.UNITY_UNIT_TEST_RESULTS_FILE}", fingerprint: true
          }
        }
      }
    }

    stage('SonarQube Analysis') {
      steps {
        script {
          // 1) Locate the MSBuild flavor of the Sonar scanner
          def scannerHome = tool 'SonarScanner_NET'
          echo "SonarScanner for MSBuild at: ${scannerHome}"

          // 2) Wrap everything in your SonarQube server context
          withSonarQubeEnv('EmuQube') {
            bat "\"${scannerHome}\\SonarScanner.MSBuild.exe\" begin " +
                "/k:\"${PROJECT_NAME}\" " +
                "/d:sonar.sourceEncoding=UTF-8 " +
                "/d:sonar.exclusions=\"**/Plugins/**/*,**/Libraries/**/*,**/Vendor/**/*,**/Samples/**,**/XR/**,**/Packages/**/*,**/ThirdParty/**/*\""

            bat "MSBuild.exe \"${UNITY_PROJECT_PATH}\\Endless Book.sln\" /t:Rebuild /p:Configuration=Debug"

            bat "\"${scannerHome}\\SonarScanner.MSBuild.exe\" end\""            
          }
        }
      }
    }
  }

  post {
    always {
      archiveArtifacts artifacts: "${env.UNITY_UNIT_TEST_RESULTS_FILE}", fingerprint: true
    }
  }
}

Preparing the Pipeline Dependencies

Configuring JDK

If we select Build Now on our pipeline and see the results. If you do this there will be the following error

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 15: Tool type "jdk" does not have an install of "JDK17" configured - did you mean "null"? @ line 15, column 13.
           jdk 'JDK17'

Notice our Jenkinsfile has a tool jdk 'JDK17', we need to create this before we reference it, so let’s do that now.

  • Click on the cog symbol at the top right to manage jenkins
  • Select Tools
  • Under JDK Installations select Add JDK, we give it the name “JDK17” to match the Jenkinsfile and leave JAVA_HOME blank, we also don’t need to check the box that says “Install automatically” as we are using our local jdk already installed.

Configuring MSBuild

We will use MSBuild to build unity.

  • Go to manage jenkins
  • Click on tools
  • Scroll down to “MSBuild Installations”
  • Click on Add MSBuild and give it a name to reference in the Jenkinsfile
  • You can point it to an installed one or install automatically
  • Click Save

Configuring SonarQube

Next we need to configure SonarQube server for jenkins. This also requires a SonarQube server to be already running. In this case I am using the community version in my local windows environment.

  • Go to manage jenkins
  • Select System under System Configuration heading
  • Scroll down to SonarQube servers
  • Select Add sonarqube and give it a name
  • Click on save

We can now add a sonar scanner

  • Go to manage jenkins
  • Select Tools
  • Select Add SonarScanner for MSBuild
  • Give it a name to match the Jenkinsfile (“SonarScanner_NET” in this case
  • You can either point it to an existing MSBuild path or select one to be installed automatically

Adding Credentials

We next need to add some credentials for unity, when running in batch mode it will still need to validate the license, personal licenses do not have a serial so we just need to add login credentials. Here is what we do

  • Go to Manage Jenkins
  • Select Credentials under Security category
  • Select global
  • Select Add Credentials
  • Use the kind dropdown and choose Username with password
  • Enter the username (email for unity despite what unity dashboard says your username is) and password. Also give it the id to match the Jenkinsfile and we have it as “unity-login” under the “Activate Unity License” stage

Running the Pipeline

The pipeline we created should be displayed in the jenkins landing page, in my case at http://localhost:9080/

We can select our pipeline below

Next we build our pipeline

Once we build you will see it running at the bottom left

Once its finished it will show green (if successful) in the progress box, it will also share the info we need on the status page such as sonar quality gateway, an xml of our unity test results etc. It will also give us a link to sonar to dig in deeper

Finally if we click on the build at the bottom left we can view more info about that specific build and also see console logs.

Conclusion

We have seen how to set up a simple jenkins pipeline for unity, there are a lot of moving parts but once this foundation is laid it can lead to more interesting options for the future, such as automating builds, using VC triggers, security scans and anything you want to automate in the pipeline!

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *