SonarQube Azure DevOps Marketplace Extension - Java Heap OOM Exception Failing Build

csharp
sonarqube
scanner
azure

(dave) #1
  • versions used (SonarQube, Scanner, Plugin, and any relevant extension)
    SonarScanner for MSBuild 4.6, Sonar Server Version 7.1.0.11001, Our project is a large (~800k lines) project with T-SQL, C# and TypeScript.

  • error observed
    java.lang.OutOfMemoryError: Java heap space

  • steps to reproduce
    Full scan with Code Coverage Analysis using VS Enterprise 2017 CodeCoverage.exe

  • potential workaround
    temporary fix: downgrade SonarQube Azure Devops Marketplace Extension back to ver 4.5.1

Our nightly (full) Sonar scan started dying a couple weeks ago. We noticed the extension had updated automatically from 4.5.1 to 4.6.0. It was discovered that TFS will automatically update modules without warning and without a way to disable the auto-updating. This is a separate issue (https://visualstudio.uservoice.com/forums/330519-azure-devops-formerly-visual-studio-team-services/suggestions/15205311-auto-update-for-onpremise-tfs-2015-marketplace-e).

The updated (4.6.0) build was dying on a Java Heap OOM exception during the Duplications analysis step of the CodeCoverage analysis. We spent a couple days trying to get it to work by modifying the SONAR_SCANNER_OPTS variable to larger sizes and trying to exclude files that might be causing the issue, to no avail. Finally we downgraded the extension back to 4.5.1 and got it working, same as before. The problem is that when the extension inevitably updates itself again, we will be in the same position.

We suspect the SONAR_SCANNER_OPTS variable wasn’t being set, despite our setting it as a system environment variable. This is in part because every time we modified SONAR_SCANNER_OPTS, the build logs always showed Final Memory: 14M/247M in the failed build summary using Debug logging. If a debug log would be valuable, we would be happy to provide.

Please let us know what we can do or if there is a fix that can be issued to make the 4.6.0 SonarQube extension stop failing with a java heap OOM exception.

Thank you!

Below is the output from the failed build:

2019-03-08T16:03:44.0854905Z 11:03:44.071 INFO: ------------------------------------------------------------------------
2019-03-08T16:03:44.0855623Z 11:03:44.071 INFO: EXECUTION FAILURE
2019-03-08T16:03:44.0856140Z 11:03:44.071 INFO: ------------------------------------------------------------------------
2019-03-08T16:03:44.0856687Z 11:03:44.071 INFO: Total time: 4:08.581s
2019-03-08T16:03:44.1571760Z 11:03:44.148 INFO: Final Memory: 14M/247M
2019-03-08T16:03:44.1577032Z 11:03:44.148 INFO: ------------------------------------------------------------------------
2019-03-08T16:03:44.1578974Z ##[error]11:03:44.148 ERROR: Error during SonarQube Scanner execution
java.lang.OutOfMemoryError: Java heap space
2019-03-08T16:03:44.1580088Z 11:03:44.148 ERROR: Error during SonarQube Scanner execution
2019-03-08T16:03:44.1580592Z java.lang.OutOfMemoryError: Java heap space
2019-03-08T16:03:44.1581672Z ##[error]at org.sonar.duplications.index.PackedMemoryCloneIndex.ensureCapacity(PackedMemoryCloneIndex.java:286)
2019-03-08T16:03:44.1582652Z at org.sonar.duplications.index.PackedMemoryCloneIndex.ensureCapacity(PackedMemoryCloneIndex.java:286)
2019-03-08T16:03:44.1583745Z ##[error]at org.sonar.duplications.index.PackedMemoryCloneIndex.insert(PackedMemoryCloneIndex.java:251)
2019-03-08T16:03:44.1584749Z at org.sonar.duplications.index.PackedMemoryCloneIndex.insert(PackedMemoryCloneIndex.java:251)
2019-03-08T16:03:44.1585846Z ##[error]at org.sonar.scanner.cpd.index.SonarCpdBlockIndex.insert(SonarCpdBlockIndex.java:74)
2019-03-08T16:03:44.1586788Z at org.sonar.scanner.cpd.index.SonarCpdBlockIndex.insert(SonarCpdBlockIndex.java:74)
2019-03-08T16:03:44.1588234Z ##[error]at org.sonar.scanner.sensor.DefaultSensorStorage.store(DefaultSensorStorage.java:475)
at org.sonar.api.batch.sensor.cpd.internal.DefaultCpdTokens.doSave(DefaultCpdTokens.java:128)
2019-03-08T16:03:44.1589323Z at org.sonar.scanner.sensor.DefaultSensorStorage.store(DefaultSensorStorage.java:475)
2019-03-08T16:03:44.1590159Z at org.sonar.api.batch.sensor.cpd.internal.DefaultCpdTokens.doSave(DefaultCpdTokens.java:128)
2019-03-08T16:03:44.1591210Z ##[error]at org.sonar.api.batch.sensor.internal.DefaultStorable.save(DefaultStorable.java:45)
2019-03-08T16:03:44.1592143Z at org.sonar.api.batch.sensor.internal.DefaultStorable.save(DefaultStorable.java:45)
2019-03-08T16:03:44.1593812Z ##[error]at org.sonarsource.dotnet.shared.plugins.protobuf.CPDTokensImporter.consumeFor(CPDTokensImporter.java:48) 
at org.sonarsource.dotnet.shared.plugins.protobuf.CPDTokensImporter.consumeFor(CPDTokensImporter.java:32)
2019-03-08T16:03:44.1594989Z at org.sonarsource.dotnet.shared.plugins.protobuf.CPDTokensImporter.consumeFor(CPDTokensImporter.java:48)
2019-03-08T16:03:44.1596519Z at org.sonarsource.dotnet.shared.plugins.protobuf.CPDTokensImporter.consumeFor(CPDTokensImporter.java:32)
2019-03-08T16:03:44.1598100Z ##[error]at org.sonarsource.dotnet.shared.plugins.protobuf.ProtobufImporter.consume(ProtobufImporter.java:69)
2019-03-08T16:03:44.1599218Z at org.sonarsource.dotnet.shared.plugins.protobuf.ProtobufImporter.consume(ProtobufImporter.java:69)
2019-03-08T16:03:44.1600651Z ##[error]at org.sonarsource.dotnet.shared.plugins.protobuf.RawProtobufImporter.accept(RawProtobufImporter.java:44)
at org.sonarsource.dotnet.shared.plugins.ProtobufDataImporter.parseProtobuf(ProtobufDataImporter.java:103)
2019-03-08T16:03:44.1602068Z at org.sonarsource.dotnet.shared.plugins.protobuf.RawProtobufImporter.accept(RawProtobufImporter.java:44)
2019-03-08T16:03:44.1602690Z at org.sonarsource.dotnet.shared.plugins.ProtobufDataImporter.parseProtobuf(ProtobufDataImporter.java:103)
2019-03-08T16:03:44.1608710Z ##[error]at org.sonarsource.dotnet.shared.plugins.ProtobufDataImporter.importResults(ProtobufDataImporter.java:74)
at org.sonar.plugins.csharp.CSharpSensor.executeInternal(CSharpSensor.java:90)
at org.sonar.plugins.csharp.CSharpSensor.execute(CSharpSensor.java:68)
at org.sonar.scanner.sensor.SensorWrapper.analyse(SensorWrapper.java:53)
at org.sonar.scanner.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:88)
at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:82)
at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:73)
at org.sonar.scanner.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:88)
at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:180)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:121)
at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:302)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:297)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:271)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:121)
at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:48)
at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:84)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:121)
at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:121)
2019-03-08T16:03:44.1612140Z at org.sonarsource.dotnet.shared.plugins.ProtobufDataImporter.importResults(ProtobufDataImporter.java:74)
2019-03-08T16:03:44.1612749Z at org.sonar.plugins.csharp.CSharpSensor.executeInternal(CSharpSensor.java:90)
2019-03-08T16:03:44.1613283Z at org.sonar.plugins.csharp.CSharpSensor.execute(CSharpSensor.java:68)
2019-03-08T16:03:44.1613809Z at org.sonar.scanner.sensor.SensorWrapper.analyse(SensorWrapper.java:53)
2019-03-08T16:03:44.1614351Z at org.sonar.scanner.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:88)
2019-03-08T16:03:44.1615019Z at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:82)
2019-03-08T16:03:44.1615557Z at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:73)
2019-03-08T16:03:44.1616116Z at org.sonar.scanner.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:88)
2019-03-08T16:03:44.1616692Z at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:180)
2019-03-08T16:03:44.1617361Z at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
2019-03-08T16:03:44.1617941Z at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:121)
2019-03-08T16:03:44.1618500Z at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:302)
2019-03-08T16:03:44.1619076Z at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:297)
2019-03-08T16:03:44.1619677Z at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:271)
2019-03-08T16:03:44.1620303Z at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
2019-03-08T16:03:44.1620879Z at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:121)
2019-03-08T16:03:44.1621409Z at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:48)
2019-03-08T16:03:44.1622039Z at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:84)
2019-03-08T16:03:44.1622602Z at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:135)
2019-03-08T16:03:44.1623229Z at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:121)
2019-03-08T16:03:44.1623796Z at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:121)
2019-03-08T16:03:44.2047129Z Process returned exit code 1
2019-03-08T16:03:44.2049647Z ##[error]The SonarQube Scanner did not complete successfully
2019-03-08T16:03:44.2050757Z The SonarQube Scanner did not complete successfully
2019-03-08T16:03:44.2132743Z ##[error]11:03:44.209 Post-processing failed. Exit code: 1
2019-03-08T16:03:44.2133743Z 11:03:44.209 Post-processing failed. Exit code: 1
2019-03-08T16:03:44.2282040Z ##[error]C:\Builds\Agent1\_work\_tasks\SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157\4.6.0\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe failed with return code: 1
2019-03-08T16:03:44.2328078Z ##[section]Finishing: Run Code Analysis

(Duncan Pocklington) #2

@dwick the root cause is almost certainly this change in the Scanner for MSBuild, which in turn is embedded inside the SonarQube Azure DevOps extension.

If you set system.debug=true for the Azure DevOps build you should see log entries for the Run Code Analysis step that refer to SONAR_SCANNER_OPTS if it is being passed to the task (i.e. it has the same effect as passing /d:sonar.verbose=true).

Did you restart the build agent service on your build machines after setting the environment variable? If I remember correctly, the build agent service caches the environment variables at startup.

Another option to try for debugging purposes is to set SONAR_SCANNER_OPTS as an Azure DevOps build variable for the failing build. All non-secret build variables are passed to the build as environment variables.


(dave) #3

Hey @duncanp!
Thanks for your speedy reply and sorry for the delay. We manually updated the extension to 4.6.0 and restarted to ensure it had the correct environment variables loaded (not using the cached values). This seems to have done the trick!
Appreciate your help!
-Dave