java.lang.IllegalArgumentException: Multiple entries with same key

AlainODea · February 28, 2019, 3:57pm

versions used (SonarQube, Scanner, Plugin, and any relevant extension)

SonarQube Enterprise 7.6.0.21501
SonarQube Scanner 3.2.0.1227
SonarQube Scanner for Jenkins 2.8.1

error observed (wrap logs/code around triple quote ``` for proper formatting)

PR-15500 shows this image:

With this text:
SonarQube Failed to check Code Quality

This is reported on the SonarQube server when I check background tasks for PR-15500:

java.lang.IllegalArgumentException: Multiple entries with same key: REDACTED:PULL_REQUEST:15500=ComponentImpl{type=FILE, status=ADDED, name='REDACTED', dbKey='REDACTED:PULL_REQUEST:15500', key='REDACTED', uuid='AWkmavhEgHOnJc9HB16f', description='null', children=[], projectAttributes=null, reportAttributes=ReportAttributes{ref=6862, path='null', scmPath='REDACTED/src/REDACTED/REDACTED.java'}, fileAttributes=FileAttributes{languageKey='java', unitTest=false, lines=31}} and REDACTED:PULL_REQUEST:15500=ComponentImpl{type=FILE, status=CHANGED, name='REDACTED', dbKey='REDACTED:PULL_REQUEST:15500', key='REDACTED', uuid='AWkmavhEgHOnJc9HB16f', description='null', children=[], projectAttributes=null, reportAttributes=ReportAttributes{ref=8896, path='null', scmPath='REDACTED/src/main/java/REDACTED.java'}, fileAttributes=FileAttributes{languageKey='java', unitTest=false, lines=62}}
	at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:150)
	at com.google.common.collect.RegularImmutableMap.checkNoConflictInBucket(RegularImmutableMap.java:104)
	at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:70)
	at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:254)
	at org.sonar.ce.task.projectanalysis.filemove.FileMoveDetectionStep.getReportFilesByKey(FileMoveDetectionStep.java:186)
	at org.sonar.ce.task.projectanalysis.filemove.FileMoveDetectionStep.execute(FileMoveDetectionStep.java:113)
	at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:77)
	at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:69)
	at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:56)
	at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:137)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:100)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:84)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:76)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:46)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

steps to reproduce

This does not happen consistently, but does happen frequently on PR scans. There are a lot of PR scans that overlap in time.

potential workaround

None known at this time. I’m seeing if rerunning the pipeline in Jenkins resolves it. It happens frequently enough that this is not a viable workaround long-term.

AlainODea · February 28, 2019, 5:04pm

For context, this has happened 50 times out of the 97 PR scans today alone. That is more than 50% of the PRs analyzed. It is disabling our ability to use SonarQube in our pull request process. This appears to be new since we upgraded from 7.4 to 7.6.

AlainODea · February 28, 2019, 5:10pm

Even more troubling is that this failure to analyze is posted as a failed commit check which blocks our merge process. Rerunning the build in Jenkins does not resolve this condition.

Julien_HENRY · March 1, 2019, 10:23am

Hi Alain,

In SonarQube 7.6 we have dropped the concept of modules on server side, and added aggregation of measures per folders. This is a complex migration, and maybe this is the cause or your issue.

Few questions:

is your project a multi-module project ?
was your PR first analyzed in 7.4, and now failing in 7.6, or is it a brand new PR?
I see you have obfuscated keys and path of components in the stacktrace. Could you please send the unobfuscated message to me privately so that I can better investigate? I talking about:

java.lang.IllegalArgumentException: Multiple entries with same key: REDACTED:PULL_REQUEST:15500=ComponentImpl{type=FILE, status=ADDED, name='REDACTED', dbKey='REDACTED:PULL_REQUEST:15500', key='REDACTED', uuid='AWkmavhEgHOnJc9HB16f', description='null', children=[], projectAttributes=null, reportAttributes=ReportAttributes{ref=6862, path='null', scmPath='REDACTED/src/REDACTED/REDACTED.java'}, fileAttributes=FileAttributes{languageKey='java', unitTest=false, lines=31}} and REDACTED:PULL_REQUEST:15500=ComponentImpl{type=FILE, status=CHANGED, name='REDACTED', dbKey='REDACTED:PULL_REQUEST:15500', key='REDACTED', uuid='AWkmavhEgHOnJc9HB16f', description='null', children=[], projectAttributes=null, reportAttributes=ReportAttributes{ref=8896, path='null', scmPath='REDACTED/src/main/java/REDACTED.java'}, fileAttributes=FileAttributes{languageKey='java', unitTest=false, lines=62}}

AlainODea · March 1, 2019, 4:35pm

is your project a multi-module project ?

It is not a multi-module project in SonarQube. It has multiple Gradle modules, but they are all scanned together in a single SonarQube scan with no module configuration.

was your PR first analyzed in 7.4, and now failing in 7.6, or is it a brand new PR?

PR-15500 (and others) were previously scanned by 7.4.

I see you have obfuscated keys and path of components in the stacktrace. Could you please send the unobfuscated message to me privately so that I can better investigate? I talking about:

SUPPORT-13257 has the details including the support information JSON. I can PM you as well.

AlainODea · March 1, 2019, 4:46pm

I don’t see how to private message you. Please let me know if the support case isn’t accessible and how to PM you.

AlainODea · March 1, 2019, 4:49pm

Our branch scans are also failing for the same reason, which means the PRs might not have a valid base and may start reporting issues from their underlying branch.

Julien_HENRY · March 4, 2019, 9:01am

Hi Alain,

Our branch scans are also failing for the same reason

Can you confirm your are talking about long living branch analysis?

It is not a multi-module project in SonarQube. It has multiple Gradle modules, but they are all scanned together in a single SonarQube scan with no module configuration.

You mean you are using the SonarScanner CLI to analyze a Gradle project? How are you passing the correct configuration for sonar.java.binaries/sonar.java.libraries? I don’t think this is important for your issue, but this is to be sure we are talking about the same thing.

AlainODea · March 4, 2019, 11:41am

Thank you for looking into this.

This is happening on both branch scans of branches configured as long-lived in SonarQube and pull request scans of pull requests targeted to be merged to those branches.

Yes. We are using SonarScanner CLI to analyze a Gradle project.

I output the source and output directories using Gradle and specified them explicitly in sonarproject.properties. I don’t like this workaround, but the Gradle Plugin runs tests and our tests take too long for our process unless they are split and run on multiple Jenkins agents in parallel.

AlainODea · March 12, 2019, 11:14am

Has there been any further insight here? These “Failed to Check Code Quality” outcomes on PRs are really impeding our release process.

dmeneses · March 14, 2019, 7:59pm

Hi Alain,

We are working with support to try to find the cause of the error.

AlainODea · March 14, 2019, 9:37pm

Thank you, Duarte

AlainODea · March 20, 2019, 10:13pm

Alexandre Frigout helped me find the cause. I was running a 7.4 server connected to the same database. It was taking some of the background tasks and conflicting with the 7.6 server.