Duplicate key message

  • IntelliJ IDEA 2020.2; SonarLint 6.5.1.43866

  • I am trying to install & configure SonarLint via Server. I followed below steps but getting errors.

  • In IntelliJ IDEA, I went to File > Settings > Tools > SonarLint > Project Settings > Selected CheckBox for Bind project to SonarQube/SonarCloud > Created a new connection with Token > Clicked on Update binding > Got below error message.

  • Error
    Server started on 64120
    Plugin ‘secrets’ embeds dependencies. This will be deprecated soon. Plugin should be updated.
    Plugin ‘swift’ embeds dependencies. This will be deprecated soon. Plugin should be updated.
    Sync quality profiles…
    Downloaded plugin list in 54ms
    [SYNC] Synchronizing analyzer configuration for project ‘com.mycompanyname’
    Downloaded settings in 52ms
    [SYNC] Fetching rule set for ‘java’
    [SYNC] Fetching rule set for ‘js’
    [SYNC] Fetching rule set for ‘kotlin’
    [SYNC] Fetching rule set for ‘php’
    [SYNC] Fetching rule set for ‘py’
    [SYNC] Fetching rule set for ‘ruby’
    [SYNC] Fetching rule set for ‘scala’
    [SYNC] Fetching rule set for ‘swift’
    [SYNC] Fetching rule set for ‘ts’
    [SYNC] Fetching rule set for ‘web’
    [SYNC] Fetching rule set for ‘xml’
    Error updating the storage for connection ‘TestSonarQube1’
    java.lang.IllegalStateException: Duplicate key message (attempted merging values Spring component-scan element has an invalid base-package attribute value and Spring component-scan element has an invalid base-package attribute value)
    at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:133)
    at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)
    at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)

Hello @ppat7046,

Thanks for raising this. By any chance would you have a more complete stacktrace ? This one looks truncated. Also could you check if there are some third-party plugins installed on the server ?

I had to revert all the way back to IntelliJ Sonarlint plugin version 6.3.1.40498, to get the error to go away and binding to work. 6.4.0.41939 and newer throw that error.

Unfortunately even though the error went away, Sonarlint didn’t properly analyze currently open files on the fly. I made a change that caused STS to report a Sonar error, but IntelliJ said there were no open files with errors. It did analyze all files.

Edit: On the fly SonarLint does work, just not for all rules. Basically though any version of the IntelliJ plugin 6.4.0.41939 and newer doesn’t work.

Hello @Morac,

Thanks for bringing more details to this topic. I would be interested to get the result of those 2 API calls on your SonarQube (from a browser would be enough):

  • api/plugins/installed
  • api/rules/search?qprofile=xml-enterprise-current-32055&activation=true&f=templateKey,actives&types=CODE_SMELL,BUG,VULNERABILITY&s=key

Could you upload 2 files with the results here please ?

I’m blocked from uploading files. Is there something specific you are looking for in those responses?
In the meantime, based on the stack trace error message, it looks like the plugin doesn’t like the fact that there are multiple “Spring component-scan element has an invalid base-package attribute value” values. Instead erroring on a duplicate value, the plugin should probably just ignore the duplicate.

    "actives": {
        "xml:Invalid_Spring_Component_Scan_Base_Package_Check": [{
                "qProfile": "xml-enterprise-current-32055",
                "inherit": "NONE",
                "severity": "INFO",
                "params": [{
                        "key": "message",
                        "value": "Spring component-scan element has an invalid base-package attribute value"
                    }, {
                        "key": "message",
                        "value": "Spring component-scan element has an invalid base-package attribute value"
                    }, {
                        "key": "expression",
                        "value": "//beans/component-scan[normalize-space(@base-package)\u003d\"PACKAGE NAME OF THE SERVICE OBJECTS\"]"
                    }, {
                        "key": "expression",
                        "value": "//beans/component-scan[normalize-space(@base-package)\u003d\"PACKAGE NAME OF THE SERVICE OBJECTS\"]"
                    }, {
                        "key": "filePattern",
                        "value": "**/spring-*.xml"
                    }, {
                        "key": "filePattern",
                        "value": "**/spring-*.xml"
                    }, {
                        "key": "filePattern",
                        "value": "**/spring-*.xml"
                    }, {
                        "key": "expression",
                        "value": "//beans/component-scan[normalize-space(@base-package)\u003d\"PACKAGE NAME OF THE SERVICE OBJECTS\"]"
                    }, {
                        "key": "message",
                        "value": "Spring component-scan element has an invalid base-package attribute value"
                    }
                ],
                "createdAt": "2016-03-02T14:01:47+0000",
                "updatedAt": "2016-03-02T14:01:47+0000"
            }
        ],

Hello Michael,

Those fields are not supposed to be duplicated. Before changing anything on SonarLint side, I want to understand what happened. What appears on this rule page in the UI ? You can find it through your XML Quality Profile

Looking at all the XML rules under Quality Profile I do see that multiple rule files contain the same exact rules, but since the SonarQube web site allows that (I can’t change it), it works with other software such as Bamboo and version 6.3.1.40498 of the IntelliJ plugin works without issue, it seems like the plugin should be able to handle this without throwing an error.

If it wants to put up a warning that’s fine, but if the server is fine with the configuration and works fine with other software (we use Bamboo), I don’t think the plugin should be failing. Maybe add a plugin option to allow duplicate keys?

SEC-XML02 - Session timeout must be configured
SEC-XML02 - Session timeout must be configured
SEC-XML02 - Session timeout must be configured

Hi Michael,

Could you please send me some screenshots of what you observed on those rules ? There is probably something to do on the SonarLint side, but I want to understand what happens on the SonarQube side

I’m not sure what you mean by what I observed. We currently use Bamboo to build our application and as part of the build it contacts the SonarQube server to run the rules. It works fine despite the duplicates. The SonarLint plugin throws the “Duplicate key message” exception and aborts.

I decided to dig into the response from the API and found that it contains 2 different JSON array objects that contain a “key” name: rules and activities.

{
    "total": 13,
    "p": 1,
    "ps": 100,
    "rules": [....],
    "actives": {....},
    "qProfiles": {....}
}

The keys in the rule section contain an XPatchCheck value which matches up to the one in the URL for the web page when I click on a Rule. As such that appears to be the primary key for a rule.

There are 13 rules with unique keys that look like

"rules": [
        {
            "key": "xml:XPathCheck_1401757088507",
            "templateKey": "xml:XPathCheck",
            "type": "CODE_SMELL"
        },
        ....
]

The activities contains parameters which contain a key, value pair. These are not unique.
The activities section contains entries that look like (I truncated the params as there were a lot of values with duplicates):

"actives": {
        "xml:XPathCheck_1401757088507": [{
                "qProfile": "xml-enterprise-current-32055",
                "inherit": "INHERITED",
                "severity": "CRITICAL",
                "params": [{
                        "key": "filePattern",
                        "value": "**/WEB-INF/web.xml"
                    }, {
                        "key": "filePattern",
                        "value": "**/WEB-INF/web.xml"
                    }, {
                        "key": "filePattern",
                        "value": "**/WEB-INF/web.xml"
                    }, {
                        "key": "filePattern",
                        "value": "**/WEB-INF/web.xml"
                    }, {
                        "key": "filePattern",
                        "value": "**/WEB-INF/web.xml"
                    }.....
                ],
                "createdAt": "2018-12-15T05:38:00+0000",
                "updatedAt": "2018-12-15T05:38:04+0000"
            }
        ],
        ....
}

As far as I can tell the key for parameters in activities do not need to be unique, but the key in the rules do. I think the problem here is the SonarLint plugin is checking both the rule and activities keys and erroring for the later even though it’s allowed.

As I mentioned, I’m currently using the 6.3.1.40498 SonarLint plugin which doesn’t have the check for duplicate keys. As far as I can tell it is working as expected. The only thing keeping me from upgrading to the current version is the duplicate key check.

Any work on fixing this? The plugin shouldn’t be enforcing uniqueness for non-unique fields.

Hi @Morac

Sorry for the late answer.

Your SonarQube database is in an inconsistent state. I don’t know how it ended up like that, possibly because of a previous migration, or a bug in an older version. I was not able to reproduce it.

We don’t want to put a hack in SonarLint to support this situation, as it may lead to other issues. The fact it was “working” (well, not crashing) previously was pure coincidence.

I think it should be possible to fix your SonarQube database by simply dropping and re-adding the custom XPath rule (don’t forget to add it back in your quality profile).

I will notify my SonarQube team colleagues to see if they want to investigate more.