C++ unit tests reporting (MS Unit Test Framework for C++)

SonarQube Server / Community Build
, , ,
1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)

    • SonarQube Developer 8.4.1, CFamily 6.11.0
    • Visual Studio Enterprise 2019
    • Azure DevOps Server 2019

  • what are you trying to achieve
    Report unit tests and code coverage to SonarQube

    Visual Studio supports the following C++ testing frameworks out of the box:

    • Microsoft Unit Testing Framework for C++
    • Google Test
    • Boost.Test
    • CTest

    As we used previous versions of Visual Studio we are using the Microsoft framework.
    I got the tests running and finaly converted the code coverage. Now I get some line coverage and coverage percentage. What is still missing is the number of unit tests.

    I read that SonarQube CFamily is only supporting CppUnit. Is this still up to date? No newer frameworks are supported? Is there any way how to convert our test results to the CppUnit format SonarQube is reading?

  • what have you tried so far to achieve this

    • Run the tests and converted code coverage.
    • Unit tests are reported in Azure DevOps Server
      # Prepare SonarQube analysis
      - task: sonarsource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4
        displayName: 'Prepare analysis on SonarQube'
        inputs:
          SonarQube: Sonarqube
          projectKey: CppPart
          projectVersion: '$(Build.BuildNumber)'
          # for mixed C++/C# solution the `SonarScanner for MSBuild` must be used!
          extraProperties: |
            sonar.projectDescription=C++ part
            sonar.exclusions=**/*.java
            sonar.projectBaseDir=$(SrcDir)
            sonar.cfamily.build-wrapper-output=$(SonarOutDir)
            sonar.cfamily.threads=4
            sonar.cfamily.cache.enabled=true
            sonar.cfamily.cache.path=$(Agent.WorkFolder)/_Cache
            sonar.cfamily.vscoveragexml.reportsPath=$(Common.TestResultsDirectory)/codeCoverage.xml

      # Build C++ solution
      - task: petergroenewegen.PeterGroenewegen-Xpirit-Vsts-Build-InlinePowershell.Xpirit-Vsts-Build-InlinePowershell.InlinePowershell@1
        displayName: 'Build solution with build wrapper'
        inputs:
          Script: |
            New-Item -Path $(SonarOutDir) -ItemType directory -Force
            Set-Location $(SrcDir)
            $(build_wrapper) --out-dir $(SonarOutDir) "$(msbuildPath)/MSBuild.exe" $(SrcDir)/Solution.sln /t:Rebuild /nologo /nr:false /t:"Clean" /p:FullRelease=true /p:platform=$(BuildPlatform) /p:configuration=$(BuildConfiguration) /p:VisualStudioVersion="16.0" /m

      - task: VSTest@2
        displayName: 'Test Assemblies C++ Unit Tests'
        inputs:
          testAssemblyVer2: '**/$(BuildConfiguration)_$(BuildPlatform)/**/*test*.dll'
          searchFolder: '$(SrcDir)'
          vsTestVersion: toolsInstaller
          codeCoverageEnabled: false # this crashes the vstest.console (bad image format)
          testRunTitle: 'C++ Unit Tests'
          otherConsoleOptions: '/platform:x64 /Logger:trx /Enablecodecoverage'
          platform: '$(BuildPlatform)'
          configuration: '$(BuildConfiguration)'

      - task: petergroenewegen.PeterGroenewegen-Xpirit-Vsts-Build-InlinePowershell.Xpirit-Vsts-Build-InlinePowershell.InlinePowershell@1
        displayName: 'Convert code coverage report'
        inputs:
          Script: |
            Set-Location $(Build.SourcesDirectory)
            & "$(codeCoveragePath)" analyze /output:$(Common.TestResultsDirectory)/codeCoverage.xml (Get-ChildItem TestResults/*/*.coverage)
2

Hi @milbrandt,

The way we currently achieve this is by performing a XSL Transform to convert the XML report(s) generated by VS Test (and Google Test etc) to a CPP Unit XML report.

Hope that’s of some help!

Cheers,

Sam

3

Thanks @Sam_Anthonisz for this idea. Are your XSLT for the transformation anywhere available? Otherwise we have to investigate for the two XML formats on our own and define the transformation again.

I assume that not only we have the wish to use Microsoft’s default testing frameworks for C++ together with SonarQube.