SonarQube/AzurePipelines doesn't pick up branch differences for .NET Framework

[JTBruch-PrimeGov (JT Bruch)]

Summary

We are adding SonarQube to our build pipeline, and we are running into a weird issue where the analysis seemingly works great for the main branches (develop, etc), however we want it to report the code coverage for the new code created on branches. With the current setup, the analysis in SonarCloud for the branch is only showing a fraction of the lines changed for new “lines to cover”.

For instance, in the screenshots below it shows that I’ve added 282 lines into our Test project to cover the Logic/Core. (In this scenario, the 575 added in the Web project is the commented lines in SwaggerConfig.cs) However the analysis only picked up 4 new lines to cover.

Versions Used:

• .NET Framework 4.6
• SonarCloud
• Azure Pipelines Agent (windows-latest)

Project Structure

• All .NET Framework 4.6
• Unit Test coverage comes in just fine (published to Azure DevOps)

SonarQube Example

pipelines.yml

steps:

- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: SonarCloudPrepare@1
  condition: ne(variables['build.sourceBranch'], 'refs/heads/develop')
  inputs:
    SonarCloud: '*****************************'
    organization: '*****************************'
    scannerMode: 'MSBuild'
    projectKey: '*****************************'
    extraProperties: |
      # Additional properties that will be passed to the scanner, 
      # Put one key=value per line, example:
      # sonar.exclusions=**/*.bin
      sonar.verbose=true
  
- task: SonarCloudPrepare@1
  condition: eq(variables['build.sourceBranch'], 'refs/heads/develop')
  inputs:
    SonarCloud: '*****************************'
    organization: '*****************************'
    scannerMode: 'MSBuild'
    projectKey: '*****************************'
    projectVersion: '$(Build.BuildNumber)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'
    codeCoverageEnabled: true
    failTaskOnFailedTests: true

- task: SonarCloudAnalyze@1

- task: SonarCloudPublish@1
  inputs:
    pollingTimeoutSec: '300'

[mickaelcaro (Mickaël Caro)]

Hi @JTBruch-PrimeGov and welcome to the community !

Have you got through the logs to see if there’s something obvisouly wrong ?

TestProjects are not counted in lines if this is rightly categorized as test project by the Scanner.

[JTBruch-PrimeGov (JT Bruch)]

@mickaelcaro Thanks for the followup.

Actually figured out the solution earlier. It was not a SonarQube issue, it just manifested on Sonar.

To those reading in the future, if you don’t have a .runsettings file set on Azure DevOps, what happens is that it generates one for you. It has a lot of default settings, and then extras. It was excluding about half the build.

The fix, was to create a .runsettings file, add it to the repo, and have the VSTest task look at it.

The new file:

<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>              
              <Exclude>
                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
              </Exclude>
            </ModulePaths>            
            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
  <RunConfiguration>
    <BatchSize>1000</BatchSize>
  </RunConfiguration>
</RunSettings>

The VSTest Task:

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\OurTestDLL.Test.dll
    codeCoverageEnabled: true
    runSettingsFile: 'VsTest.runsettings'