I was wondering if SonarQube Scanner provides a “incremental” Analysis.
My project is growing permanently and we currently have nearly 90 Modules with over 160k LOC (in one project). The analysis takes over 25 Minutes (using SonarQube Scanner).
Regarding the analysis of Git Feature Branches we are thinking about the Developer Edition but such a long running Analysis is a NoGo for our CI Jobs, especially when only a few files have been modified.
I already tried to keep the “.scannerwork” directory with no time improvement and googled for an incremental configuration but only found out that sonar.analysis.mode=incremental is deprecated for a long time now.
We do not currently support incremental analysis as it can result in less accurate scanning results. This has come up quite regularly, especially in the context of branches and pull requests, and internally there is a lot of discussion around this, but I don’t think we have anything to share at this time.
That said, 25 minutes on 160k LOC seems a little long (although that’s not exactly scientific).
Which side of the analysis is taking so long – the actual scanning or the background task on the SonarQube Server’s side? (If the former, how long is the background task taking to process?)
What version of SonarQube and relevant plugins are you running?
Are you using any plugins that run other analysers (PMD, Checkstyle, Findbugs, etc.)? These are often the cause of lengthened analysis tasks and is rather out of our control.
If you have logs to share from analysis, preferably with debug mode enabled, we might be able to do a sanity check.
thanks for the fast reply.
The scanning takes the 25 Minutes.
Background Task on SonarQube is finished in less than 2 minutes, so this is OK.
We’re using SonarQube 7.3 and FindBugs Plugin 3.8.0.
I will check if we’re currently using Rules from the FindBugs Plugin so that I can remove it and retest if it’s faster then.
How can I provide the logs to you if the deactivation does not have the desired effect?
You should be able to spot in your logs how long FindBugs is taking. If indeed Findbugs is the cause of long-running analysis, that is something to take up with the plugin developer (if not Findbugs itself, as I think the plugin mostly just wraps Findbugs).
That said, SonarJava is excellent (527 rules and counting), in my opinion.
Perhaps create a second project and assign it the default Sonar Way quality profile and scan your code using that new project key? It would give you an idea of what performance in SonarQube is expected to be like by our standards. You might even find it suitable to use instead of your existing QP.
Edit: Had to strikethrough, see below
In the event you still wish to share logs, we accept them zipped up and attached to this topic.
An update. My colleague @ganncamp informed me that when the Findbugs plugin is installed on an instance, it will always run and evaluate all rules, before parsing out the ones enabled in your Quality Profile, so testing performance might not be so simple. It looks like an existing issue exists on the sonar-findbugs repo without much traction: https://github.com/spotbugs/sonar-findbugs/issues/72
I also forgot to mention that SonarJava does support the import of Findbugs reports (see: MMF-1300) so perhaps it would be possible to stop using the Findbugs plugin, pull out the Findbugs rules from your QP, and run/configure Findbugs seperately and use our native features to import the reports. Just a thought!
So you said that you do not support incremental results because they can be less accurate. I am currently working for a company that is transitioning legacy code bases to devops environments with sonarqube. One of my projects is ~250k LOC and takes around a half an hour to be scanned (Contains thousands of errors).
I understand accuracy of results is important, but I think it would be great to provide an incremental scan that could be used by specific build processes, results of which to discard so that they do not mess up the persistent full scan results.
This would allow developers to quickly deploy code to a dev or qa environment and have some sonarqube quality information restricting the full scans to the staging production environments.
You guys have developed a great piece of software but devops is all about streamlining the build and deploy process as well as quality control. Right now sonarqube is conflicting with the first goal.
Even though incremental scan can be less accurate, I’m OK to take the risk. It is better than ~15min scan time that we have now, which makes wastes developers time on waiting (they cannot proceed to merge until the pipeline is successful).
An option to disable expensive checks or to allow incremental scanning would be at least something. This way we’ll be able to do incremental on development branches and full on release branches.
Otherwise product becomes unusable, because codebase will only grow as well as the time taken by analysis.
So far, we used the now removed “Preview” analysis mode with Sonar 6. This mode took less than a minute for PRs on our 250k LOC codebase. After upgrading to Sonar 8 and enabling PR analysis, build times increased by 5 minutes because every build scans the whole codebase. Analysis now takes more time than the whole rest of the CI build including more than 10 thousand tests.
We spent a lot of time to improve our CI builds to get very fast feedback. Preview analysis mode was perfect for this. I think you should reconsider and let users opt-in to incremental analysis again.
Sorry for resurrecting a thread so old, but I read it some time ago and the title is still very inviting.
So in the meantime you released incremental analysis for Java PRs on some SonarQube editions.
Any insight into whether it will be available on Community Edition?
Hello, our company recently upgraded to Data Center Edition Version 9.8 and I am trying to be one of the few to implement this feature “Incremental analysis for Java PRs” in our project build. Can you please point me to the documentation for this or any implementation notes on how can be incorporated in gradle/Jenkins builds?
Thanks,
Mak
It looks like an existing issue exists on the sonar-findbugs repo without much traction: 
That’s great news (and this surely means you will buy our Developer Edition, right? No pressure. 
)
