CI based analysis vs Automatic one for Java/Maven Project

SonarQube Cloud
1

Hi,

I’m experiment Sonarcloud for our Open source Leshan project. This is a Java/Maven project.

Here is current topic about that.
Here is the sonarcloud page.

We currently use Automatic Analysis but I understand the recommended way should be to use CI-based Analysis with corresponding maven plugin.

Do you confirm, I should move to automatic to CI-based analysis ?

I think I understand how to configure the maven project.
I guess I will need to call the maven goal in a Github Action or Jenkins Build manually right ?

What is less clear to me is :
Does SonarCloud push comment on PR automatically like with automatic mode . ?

2

Hi,

Yes, per the docs you’ll get a better, more thorough analysis if you switch to managing it yourself.

Yes. Make sure this is after a build, though.

You’ll need to make sure your CI pipeline runs for both branches and PRs.

 
HTH,
Ann

3

Thx a lot for you answer :folded_hands:

Just in case there is an issue in the documentation (I’m not really sure of that) :

I looked at the list of rules which is supposed to be NOT supported by automatic mode.

Rules that belong to this list are not supported because the results that they currently produce are not accurate enough (see the line with JAVA_CHECKS_NOT_WORKING_FOR_AUTOSCAN)

Some rules names looks like to errors I saw in sonar for my project which is currently using automatic mode.

A non exhaustive list, I pretty sure I saw :

  • AnonymousClassShouldBeLambdaCheck
  • CombineCatchCheck
  • ClassWithOnlyStaticMethodsInstantiationCheck
  • VolatileNonPrimitiveFieldCheck

Is this list up to date ? or maybe this is doesn’t concern automatic mode for Maven project ? (maybe this is only for Gradle or Simple Java project)

4

Hi,

Thanks for reporting this. I believe it’s probably just a case of mislabel. It’s not that they don’t run; it’s that they’re not accurate.

 
HTH,
Ann

5

It would be great to have more precise information in documentation because hard to take decision with just “some rules are inaccurate”.

6

Hi,

IMO it’s already pretty clear (emphasis mine):

Rules that belong to this list are not supported because the results that they currently produce are not accurate enough

 
Ann

7

It is said “not supported”.

At first sight I understand that mean that rules are not run at all. (from a user point of view this will be easier)

Reading you, I understand now that “It’s not that they don’t run; it’s that they’re not accurate.”

IMHO, this sounds not so clear. (but my level in English I could be the issue. This is not native language)

Anyway, concerning not accurate enough is very unclear to me. What does that means concretely for each rules ?
From a user point of view, It seems to me that it’s hard to decide between automatic mode and move to CI-Based one based so evasive information.

In a general way, Sonar provide many different way to analyze with more or less set-up more or less constraints but that sounds hard for a user to make choice because too evasive description of differences.
(see also Launching maven plugin separately)

Anyway, that’s just user feedback maybe not representative. :person_shrugging:
So feel free to ignore it, if you think that current documentation is clear enough. :slightly_smiling_face:

8

Hi,

It’s going to vary, rule to rule. Expect both false positives and false negatives.

Ehm… Not so much. If you’re happy with a basic analysis, and with knowing that you’re probably missing some interesting stuff, then stick with automatic analysis. If you want as thorough an analysis as possible, then switch to CI-based.

 
HTH,
Ann

10

Ok but the documentation say that CI-based is the recommended way.
And when I tried to understand why this is better/recommended, I just get : “there is some stuff which is better”

Generally I base my choice on more concrete information.
E.g. I will not buy a premium edition of something if this is just said “like the classic edition but with some stuff which work better on premium”.

That sounds obvious to me but visibly we disagree :slightly_smiling_face:

Anyway as I said that was just a (maybe not so good) feedback :person_shrugging:

11

I’m aware that I ask complicated (or maybe even painful) questions.

Sorry about that :woman_bowing:.

2 more questions relative to that topic.
I know I sonar can be used to :

Is this works only with CI-based mode ? or does it also works with automatic mode ?

12

Hi,

Not to worry. That’s what we’re here for.

Yes.

Theoretically, you could add the parameters to pick up 3rd-party reports to your .sonarcloud.properties file, but automatic analysis wouldn’t know to run the tools first.

 
HTH,
Ann