We are currently evaluating whether using SonarCloud is worth the money for our company. We have a (rather large) Java Multi-Module maven project and want to have feedback within the PRs. We use Azure DevOps.
Unfortunately, I am a bit disappointed about the performance. We have the problem that when we evaluation everything it takes over one hour to analyse. This is not acceptable for us for a Pull Request integration. So I have a few questions:
How does the analysis technically work? I was hopeing it would scan only the diff or at least restrict it to the changed modules, but it seems it always analysis our whole multi-module project. Is there maybe some way to restrict that, especially because normally a commit always affects at most two of the modules
Is there any way to speed up the analysis? We have the problem that our Azure DevOps pipelines are restricted to 60 minutes runtime, so we are hitting this limit constantly.
Is there any way to figure out which rules take the most time, because maybe we could live with didabling a few of them? Also, in the logs it shows the ten most time-consuming files, but is there a analysis to show why they take so much them, i.e. which rules takes so long to evaluate for them?
Is it expected that the analysis of a roughly 1 Million LOC projects like ours takes an hour or is there something really going wrong here?
Any help would be appreciated, especially because if we can’t speed it up we will have a hard time arguing with management that this is worth the cost for us. Being faster would be a big help.
Hi Dirk,
Thanks for your feedback, it legitimates our current work on the performance subject. The java analyzer team is currently working on two related topics. It will have a huge impact on big monolithic repositories that can’t be analyzed by fragment:
Incremental Analysis of PR (the development is in progress)
Shared semantic between analyzed files (it should be on SonarCloud beginning of next week)
So if you want fast PR analysis you will need to wait some weeks, but meanwhile, you should see the analysis time decreasing from 60 minutes to 45 minutes next week. It should be a good start.
Hi @dirkkirsten,
Did you see some performance improvement in April compared to march? Even if the work about incremental analysis of PR is not finished, I would be interested to know if our efforts related to performance have already had a good impact on your projects.
Sorry to disappoint, but I don’t see an immediate impact. So to be able to include Sonar in our pipeline we restricted the scope on what files to analyse drastically ( think now it is only about 700k LOC) and this is the graph I see in our Azure DevOps pipelines (the orange bar is the analyze sonar task):
As you can see it basically stays the same from 30th of March till today, around 1300 seconds.
I will try the next week how it behaves when we analyze our whole project, I would expect (and hope) that it improved the runtimes there. However, I have to change our pipelines for that a bit and can’t really do a compare here, because we didn’t included it in our PR analysis, so we have no hard numbers (just my statement from above that it took over one hour)
Hi @dirkkirsten,
I tried to compare the orange surfaces pixel per pixel but I also see no improvement
And for now, our effort to support incremental analysis is targeting the next SonarQube 9.5, and only after SonarCloud. So meanwhile, it would be faster to try to split the analysis by modules and only analyze the modified or impacted ones.
Yes, but is there a way how to split it up in modules? The problem is that we have one large multi-module project, so the builld and PRs always work on the whole thing. I would love to be able to then split it up that Sonar only analyzes the changed modules (because usually at most 3 modules are affected by a single PR), but is there any automated way to do so?
We thought about analyzing what files changed in the PR and then figuring out what modules are impacted and then to include only those modules in the analysis by setting sonar.exclusions appropriately. Is there any easier way? Otherwise we would need some scripting to do that (which is of course possible, but we have to find the time to do that).
If you have a git repository with all modules depending on snapshot versions of each other, I don’t know how to split the build.
But, if some modules have their own release cycle, no changed files, and no changed dependencies then you can skip the build and analysis of those modules.
I do not recommend using sonar.exclusions. Our work on incremental analysis shows it is not so simple. If a file MyClass.java has not been changed but its related MyClass.class has changed because of a depending class method signature change, then the analysis is impacted.
In short, if you can only split the analysis if you can split the build.
@Zegar This topic has evolved a lot in the last two years. If you’re using SonarQube v9.9+ or SonarCloud, incremental analysis is supported in the context of pull requests.
