The SonarScanners download only required 3rd-party plugins

Dear Plugin Maintainers,

SonarQube 10.4 will come with a feature to only download Sonar analyzers and 3rd-party plugins when they are really required by the Scanner. To make this feature work, each analyzer or 3rd-party plugin should declare the list of languages on which they expect to raise issues through a new MANIFEST property called Plugin-RequiredForLanguages.

At the Scanner level, it will behave like this:

  1. when the property is not set by the plugin, the plugin will be downloaded whatever the content of the project (like it is the case today with SonarQube < 10.4)
  2. when the property is defined and when there are files corresponding to the language declared by the plugin, the plugin will be downloaded
  3. when the property is defined and when there are no files corresponding to the language declared by the plugin, the plugin won’t be downloaded by the scanner

This is great to save network bandwidth and speed up the bootstrap of the scans. As a side effect, the logs will also be cleaner with fewer “nothing to do” logs for plugins that really have nothing to perform on the repository content.

In SonarQube 10.4, this feature will be disabled by default to give you, plugin maintainers, time to adjust the configuration of your plugins. It will be activated by default with SonarQube 10.5.

For plugins that have a dependency on a base analyzer provided by default with SonarQube (for example, a plugin to add rules or reports to an existing language), it’s mandatory to add to the MANIFEST the property Plugin-RequiredForLanguages to avoid a hard failure.

Let’s take the example of sonar-xyz which provides additional rules for Java.

  • a user scans a repository of only Python code
  • sonar-xyz is downloaded because it doesn’t declare the new property so it is downloaded from the server at each scan (#1)
  • sonar-java is not downloaded because there are no .java files in the repo to scan (#3)
  • analysis errors-out because a NoClassDefFoundError is thrown since sonar-xyz has an unsatisfied dependency on sonar-java, which wasn’t downloaded

In order to avoid that you need to:

  • upgrade sonar-packaging-maven-plugin to version 1.22.0.705
  • add java to the configuration of sonar-packaging-maven-plugin where obviously “java” is replaced by the language your plugin is dealing with.
  • by adding the property <requiredForLanguages> to the configuration of sonar-packaging-maven-plugin, Plugin-RequiredForLanguages will be added to the MANIFEST
  • can take multiple values like this js,ts,css,web,yaml

Regards
Alex

7 Likes

Hi,

As a plugin mantainer, I’m interested in understanding how this feature impacts third-party plugins that do not depend on official SonarSource plugins.

For instance, consider the scenario where we have two plugins: a main “sonar-mylang” and a “mylang-custom-rules” that contributes extra rules for the same language.

My questions are:

  1. Does this feature work with this type of 3rd-party plugin?

  2. You mentioned that this feature will be disabled by default in version 10.4. Is there any option to enable the new behavior now for testing purposes?

  3. Should the Plugin-RequiredForLanguages property be added to both plugins?

  4. Are there any other specific changes required in the “main” plugin to support this new feature?

2 Likes

Hello,

Thanks for your valuable questions.

  1. Yes, it should work with your two plugins and both plugins should have the Plugin-RequiredForLanguages metadata

  2. I forgot indeed to mention how to activate the feature with SonarQube 10.4 so you can test it before the release of SonarQube 10.5
    You should activate the feature at the instance level in Administration > General > Performance

  3. Yes, both plugins should have the Plugin-RequiredForLanguages metadata

  4. You should make sure that your main plugin defines the suffixes of files you want to scan following this format: sonar.language.file.suffixes

Alex

3 Likes

A small suggestion from my side: I think this requirement should be documented in Plugin basics.

I was using “sonar.pluginKey.file.suffixes” in a plugin where the plugin key was different from the language key, and it worked fine until SonarQube 10.4. Now, even with the “Analyzers loading optimization” disabled, the files are not indexed anymore. Enabling the verbose logging shows “‘src\file.ext’ indexed with no language”. Changing the name of the propery fixes the issue, as expected.

I think that clarifying this requirement in the docs would help other developers too. :smile:

1 Like

Hi,

This means to prevent a download by scanner in any case for those plugins that are not relevant for analysis like i.e. monitoring or reporting plugins i must use something like that ?

Plugin-RequiredForLanguages: shouldneverhappen

Presumably the non-language-specific plugins for reporting, monitoring etc. have simply been forgotten !?
The ticket [SONAR-19996] - Jira - which was created because of
Restriction of the download of scanner plugins to those that are essentially needed has been closed as duplicate.

So the question is, how should plugin maintainers proceed if their plugin is not language-specific ?
This is also important because Sonarqube 10.5 will bring further changes.

Unfortunately the documentation has nothing on this.

Gilbert

Hi,

The main focus of this was to stop downloading unneeded language plugins. It really wasn’t targeted at things like monitoring since… we don’t bundle anything like that. :stuck_out_tongue:

But yeah. If you really want, you could kludge it by getting the maintainers to declare a fake language in the pom (<requiredForLangauges>foobar</requiredForLanguages>).

But again, our working assumption was that non-language-specific plugins would be needed for all languages, not that they would be needed for none.

The only change it will bring in this area is that the property will be on by default, rather than off.

We turned it off by default in 10.4 to give plugin maintainers a chance to catch up. Because if you have it on, and you’ve got non-compliant plugins - let’s say an old version of FindBugs (they’ve already updated IIRC) - then when you analyze a JS-only project, analysis errors out on the FB plugin’s missing Java dependency.

This is also, BTW, why we opened an issue on every language-dependent plugin. We’ve set their compatibility to end at 10.4, unless they publish an update to avoid these analysis crashes.

 
Ann

1 Like

Hi Ann,

thanks for clarifying,
So, even if the (generic) Jira ticket [SONAR-19996] - Jira has been created (and got closed as duplicate !?) because of
Restriction of the download of scanner plugins to those that are essentially needed which mentioned monitoring and reporting plugins, it’s ok if it works that way.

Gilbert

Hi Gilbert,

I wouldn’t say “because of”. In fact, this topic is something we’ve poked at a number of times without a satisfactory solution.

As a company, we believe deeply in baby steps (altho this step was actually pretty big), so even if it’s not perfect (e.g. monitoring plugins), it delivers a lot of value.

 
Ann

1 Like