Different results when using Sonarscanner on different servers

SonarQube Server / Community Build
, ,
1

We are using Jenkins with server agent server running SonarScanner and SonareScanner for MSBuild. The older agents are running Windows 2016 and we’re currently upgrading (replacing) with new Windows 2019 servers. The servers seem to be nearly identical other than the Operating System. However, when jobs run Sonar scans for a particular application they get different results. The older servers run fine and scans complete successfully with code coverage under the threshold. However, the new servers all get error results and ultimately fail due to very low (~50%) coverage. Same code, same version scanners, same configurations, and the same installs of Maven, Java, and other software.

We did notice a few discrepancies that we’ve since resolved. First, Visual Studio configs on the servers were slightly diferrent, so we resolved that. Also, older .Net versions were missing, so we resolved that. Still same results.

Looking for advice on what might cause this issue and how best to continue to address towards getting consistent results.

Here’s an example of the scan we run (names are changed to protect the innocent). This is run on Jenkins agents:

SonarScanner.MSBuild.exe" begin /k:“APP” /v:“4.24.10.59” /d:sonar.branch.name=“A” /d:sonar.scanner.skipJreProvisioning=true /d:sonar.cs.nunit.reportsPaths=“./NUnitResults.xml” /d:sonar.cs.opencover.reportsPaths=“./opencover.xml” /d:sonar.cs.vstest.reportsPaths=“F:\Jenkins\ws\APP-Build\app-branch\59\TestResults*.trx” /d:sonar.exclusions=“/CSS/Bootstrap*//,/Content//,/*.js,/*.JS”

3

Hey there.

The two best things you can do to troubleshoot are:

  1. Check the logs for coverage import. Do they indicate that the same coverage report (or a coverage report with the same number of files) is being imported?
  1. Check the coverage reports themselves. Any differences?

SonarQube just reads the report it’s given, so typically the issue is the report itself is not being generated the same way (different tests, some tests failing, etc.)