MR scanning breaking after upgraded to SonarQube Developer version 9.5

mmadeira · June 15, 2022, 1:46pm

Running the same exact MR branch after upgraded to 9.5.
It does not break for any other kind of branches.

Anything I can do other than go back to 9.4?

Thanks.

INFO: SonarScanner 4.7.0.2747

INFO: Java 11.0.1 Oracle Corporation (64-bit)

INFO: Linux 4.15.0-180-generic amd64

INFO: Analyzing on SonarQube server 9.5.0.56709

Jenkins Linux with GitLab Branch Source 625.v85cf3a_400cfe

INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 16.396s
INFO: Final Memory: 27M/97M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
java.lang.UnsupportedOperationException
	at org.sonar.java.model.GeneratedFile.status(GeneratedFile.java:201)
	at org.sonar.java.SonarComponents.fileCanBeSkipped(SonarComponents.java:476)
	at org.sonar.java.model.VisitorsBridge.scanWithoutParsing(VisitorsBridge.java:179)
	at java.base/java.util.stream.Collectors.lambda$partitioningBy$62(Collectors.java:1381)
	at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1675)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
	at org.sonar.java.ast.JavaAstScanner.scanWithoutParsing(JavaAstScanner.java:80)
	at org.sonar.java.JavaFrontend.scan(JavaFrontend.java:145)
	at org.sonar.plugins.java.JavaSensor.execute(JavaSensor.java:113)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:64)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.scan.SpringModuleScanContainer.doAfterStart(SpringModuleScanContainer.java:81)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
	at org.sonar.scanner.scan.SpringProjectScanContainer.scan(SpringProjectScanContainer.java:392)
	at org.sonar.scanner.scan.SpringProjectScanContainer.scanRecursively(SpringProjectScanContainer.java:388)
	at org.sonar.scanner.scan.SpringProjectScanContainer.doAfterStart(SpringProjectScanContainer.java:357)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
	at org.sonar.scanner.bootstrap.SpringGlobalContainer.doAfterStart(SpringGlobalContainer.java:135)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Dorian_Burihabwa · June 15, 2022, 4:24pm

Hey @mmadeira,

Thank you for reporting the issue. It looks like the analysis is breaking when trying to decide what to load from the SonarQube cache.

To help us solve the issue, can you help by answering these questions:

Is the analysis failing completely?
Are there JSP files analyzed as part of the Merge Request?

In the meantime, I suggest that you add the following parameter to your analysis: sonar.analysisCache.enabled=false.

mmadeira · June 15, 2022, 4:52pm

Below will can see where it breaks compared to using 9.4.
Yep, we have many JSP files.
Thank you.

With 9.5
INFO: SonarScanner 4.7.0.2747
INFO: Java 11.0.1 Oracle Corporation (64-bit)
INFO: Linux 4.15.0-180-generic amd64
INFO: User cache: /home/jenkins/.sonar/cache
INFO: Scanner configuration file: /home/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube_Scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/jenkins/workspace/inception-pipeline_MR-2398/sonar-project.properties
INFO: Analyzing on SonarQube server 9.5.0.56709
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Load global settings
INFO: Load global settings (done) | time=138ms
INFO: User cache: /home/jenkins/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=93ms
INFO: Load/download plugins (done) | time=296ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Process project properties (done) | time=1ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=2ms
INFO: Base dir: /home/jenkins/workspace/inception-pipeline_MR-2398/ProjectRoot
INFO: Working dir: /home/jenkins/workspace/inception-pipeline_MR-2398/ProjectRoot/.scannerwork
INFO: Load project branches
INFO: Load project branches (done) | time=104ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=189ms
INFO: Load branch configuration
INFO: Detected branch/PR in 'Jenkins'
INFO: Auto-configuring pull request '2398'
INFO: Load branch configuration (done) | time=6ms
INFO: Auto-configuring with CI 'Jenkins'
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=120ms
INFO: Load active rules
INFO: Load active rules (done) | time=3245ms
INFO: Load analysis cache
INFO: Load analysis cache | time=29ms
INFO: Pull request 2398 for merge into master from feature-tests-AssignmentManagerOrderColumnProvider
INFO: Load project repositories
INFO: Load project repositories (done) | time=105ms
INFO: SCM collecting changed files in the branch
INFO: Merge base sha1: 3be37b40edb237a870bd09df27eab0fbbef08b7e
INFO: SCM collecting changed files in the branch (done) | time=113ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Excluded sources: **@build-ws**, **/@tmp**, **/releng/**/*, **/pom.xml, **/bundle/**, **/bundle/Upgrade*.java, **/target**
INFO:   Excluded sources for coverage: **/*Test*.java
INFO: 4197 files indexed
INFO: 7283 files ignored because of inclusion/exclusion patterns
INFO: 14629 files ignored because of scm ignore settings
INFO: Quality profile for css: Sonar way
INFO: Quality profile for java: Inception
INFO: Quality profile for js: Sonar way
INFO: Quality profile for json: Sonar way
INFO: Quality profile for jsp: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: Quality profile for web: Sonar way
INFO: Quality profile for xml: Sonar way
INFO: ------------- Run sensors on module inception
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=43ms
INFO: Sensor JavaSensor [java]
INFO: Configured Java source version (sonar.java.source): none
INFO: JavaClasspath initialization
INFO: JavaClasspath initialization (done) | time=828ms
INFO: JavaTestClasspath initialization
INFO: JavaTestClasspath initialization (done) | time=3ms
Jun 15, 2022 11:08:49 AM org.apache.jasper.servlet.TldScanner scanJars
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
WARN: Some JSP pages failed to transpile. Enable debug log for details.
INFO: The Java analyzer is running in a context where unchanged files can be skipped. Full analysis is performed for changed files, optimized analysis for unchanged files.
WARN: Invalid character encountered in file /home/jenkins/workspace/inception-pipeline_MR-2398/ProjectRoot/plugins/com.rossvideo.inception.wires/src/com/rossvideo/inception/wires/parser/ANPA1312Parser.java at line 82 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 18.179s
INFO: Final Memory: 28M/114M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution

------------------------------------------------------------------------------------
With 9.4
------------------------------------------------------------------------------------
INFO: Scanner configuration file: /home/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube_Scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/jenkins/workspace/inception-pipeline_MR-2398/sonar-project.properties
INFO: SonarScanner 4.7.0.2747
INFO: Java 11.0.1 Oracle Corporation (64-bit)
INFO: Linux 4.15.0-180-generic amd64
INFO: User cache: /home/jenkins/.sonar/cache
INFO: Scanner configuration file: /home/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube_Scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/jenkins/workspace/inception-pipeline_MR-2398/sonar-project.properties
INFO: Analyzing on SonarQube server 9.4.0
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Load global settings
INFO: Load global settings (done) | time=76ms
INFO: User cache: /home/jenkins/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=47ms
INFO: Load/download plugins (done) | time=245ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Process project properties (done) | time=0ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=1ms
INFO: Base dir: /home/jenkins/workspace/inception-pipeline_MR-2398/ProjectRoot
INFO: Working dir: /home/jenkins/workspace/inception-pipeline_MR-2398/ProjectRoot/.scannerwork
INFO: Load project branches
INFO: Load project branches (done) | time=15ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=20ms
INFO: Load branch configuration
INFO: Detected branch/PR in 'Jenkins'
INFO: Auto-configuring pull request '2398'
INFO: Load branch configuration (done) | time=4ms
INFO: Auto-configuring with CI 'Jenkins'
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=46ms
INFO: Load active rules
INFO: Load active rules (done) | time=1179ms
INFO: Pull request 2398 for merge into master from feature-tests-AssignmentManagerOrderColumnProvider
INFO: Load project repositories
INFO: Load project repositories (done) | time=130ms
INFO: SCM collecting changed files in the branch
INFO: Merge base sha1: 3be37b40edb237a870bd09df27eab0fbbef08b7e
INFO: SCM collecting changed files in the branch (done) | time=88ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Excluded sources: **@build-ws**, **/@tmp**, **/releng/**/*, **/pom.xml, **/bundle/**, **/bundle/Upgrade*.java, **/target**
INFO:   Excluded tests: **/*Test.class
INFO:   Excluded sources for coverage: **/*Test*.java
INFO: 4197 files indexed
INFO: 7283 files ignored because of inclusion/exclusion patterns
INFO: 14629 files ignored because of scm ignore settings
INFO: Quality profile for css: Sonar way
INFO: Quality profile for java: Inception
INFO: Quality profile for js: Sonar way
INFO: Quality profile for json: Sonar way
INFO: Quality profile for jsp: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: Quality profile for web: Sonar way
INFO: Quality profile for xml: Sonar way
INFO: ------------- Run sensors on module inception
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=20ms
INFO: Sensor JavaSensor [java]
INFO: Configured Java source version (sonar.java.source): none
INFO: JavaClasspath initialization
INFO: JavaClasspath initialization (done) | time=846ms
INFO: JavaTestClasspath initialization
INFO: JavaTestClasspath initialization (done) | time=3ms
Jun 09, 2022 12:27:58 PM org.apache.jasper.servlet.TldScanner scanJars
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
WARN: Some JSP pages failed to transpile. Enable debug log for details.
INFO: Using ECJ batch to parse 1803 Main java source files with batch size 210 KB.
INFO: Starting batch processing.
INFO: Cannot determine whether the context allows skipping unchanged files: canSkipUnchangedFiles not part of sonar-plugin-api. Not skipping. org.sonar.scanner.sensor.ModuleSensorContext.canSkipUnchangedFiles()
WARN: Invalid character encountered in file /home/jenkins/workspace/inception-pipeline_MR-2398/ProjectRoot/plugins/com.rossvideo.inception.wires/src/com/rossvideo/inception/wires/parser/ANPA1312Parser.java at line 82 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
WARN: Fallback to file by file analysis for 10 files
Status ERROR: org.eclipse.jdt.core code=4 Could not retrieve declared methods org.eclipse.jdt.internal.compiler.problem.AbortCompilation: Pb(324) The type com.rossvideo.common.mos.MosChannel cannot be resolved. It is indirectly referenced from required .class files
INFO: 100% analyzed

Dorian_Burihabwa · June 16, 2022, 11:50am

Thanks for the quick feedback @mmadeira.
We are working on getting a fix readily available.

Did you manage to get the analysis going through on 9.5 by disabling the cache with the extra analysis parameter?

mmadeira · June 16, 2022, 5:58pm

Yes, I did.
I also suspected the cache, but I did not know the parameter to use.
Thank you.