I’m using SonarCloud to analyze one of our company’s C# WinForms app. After turning on the 3 tasks in the pipeline, the build time increased from 2 mins to 18 mins.
The task to run the code analysis runs for a really long time, sometimes it finishes after 40 minutes or so, sometimes it just hangs and eventually times out with this error:
##[error]java.lang.OutOfMemoryError: Java heap space
P.S. we started using SonarCloud over a year ago, but at some point around may last year we had to disable it because builds suddenly started to take forever. We’re re-enabling it now and it still doesn’t work.
Our build runs in a Hyper-V VM, the VM is used only for this build and has 4 virtual processors and 6GB of dedicated RAM (not dynamically allocated). I could increase resources but that should be enough to build an application.
Be sure to spot the difference between your actual physical resources on the machine, and the JVM heap space, which is 256Mb by default, if you haven’t changed anything.
Hi, I increased the heap size of the jvm to 2GB and I didn’t get the out of memory exception any more. However, the build still takes ages to compile and analyze. The whole process takes ~20 minutes with SonarCloud disabled. After I enable it it takes over one hour.
Any feedback on this? I’ve been silent for a while as I tried many other approaches, for the moment I worked around it but it still takes a lot of time to analyze.
What I’ve tested?
Changed the analysis type from MSBuild to stand alone. This broke the analysis as C# code was no longer being analyzed, probably that’s the expected behavior so this change was reverted.
Fixed over 2k (~25%) code smells and vulnerabilities to see if reducing the number of issues also reduces the build time. It didn’t.
Upgraded .NET from 4.5 to 4.7 because I’ve read that code analysis is faster on that version. It helped but marginally.
How I worked around it?
I had to split my build in two and run them in parallel, each part takes approximately 25 minutes
One part compiles without code analysis (which goes really fast) and then runs all the unit and integration testing
The other part then, only does the code analysis and publishing to SonarCloud
Ultimately I don’t think this works as a final solution because it forces me to have multiple agents running in parallel which has a cost, and it needs to be done for each step of the application life cycle.
Thanks for sharing your insights with the community.
It would really help us to tell us:
how long the build step takes (when most of the analysis is done)
how long the END step of the analysis takes (when the taint analysis is done)
share the logs of the END step of the analysis (we are particularly interested in Sensor CSharpSecuritySensor [security] (done) | time=)
Some background: during the END step, an advanced security analysis is done (taint analysis) which loads the representation of the full program in memory trying to find when un-sanitized data from user inputs (“sources”) reaches sensitive APIs (“sinks”) such as SQL queries or file system access. This is memory and CPU intensive.
We need more info from you to find out if the security analysis is the cause of the increase of analysis time.
So it’s clear that it’s the actual build + analysis time that takes most of the time, not the security taint analysis which is done in the END step.
To narrow down the perf hotspots during the build, could you please run the build with
msbuild /p:reportanalyzer=true /v:d > build.log
and give us the results?
This way we can identify if it’s a particular rule or set of rules that are taking longer, and then by disabling the rules with problems, the build time should decrease.
Hi, the build task in Azure DevOps doesn’t allow me to specify the forward to build.log. I’m now running the build using create log file. Since these logs might contain more information than what you need, and since I might not be able to clean up the whole log for private info, please let me know what you’re looking for and I’ll provide you with those extracts.
P.S. I couldn’t find the logfile after enabling create log file and I realized there’s no documentation on MS docs. Is there any other way I can provide you with the data?
Sorry, you can just run the command w/o output redirection msbuild /p:reportanalyzer=true /v:d
and get the output from the console output in Azure DevOps. That command with output redirection was good if you were doing it on your own machine.
Indeed, you’re right - we’re interested in the time-per rule, something like the below (but for all rules )
Total analyzer execution time: 893.913 seconds.
NOTE: Elapsed time may be less than analyzer execution time because analyzers can run concurrently.
Time (s) % Analyzer
893.913 100 SonarAnalyzer.CSharp, Version=7.13.0.0, Culture=neutral, PublicKeyToken=c5b62af9de6d7244
448.313 50 SonarAnalyzer.Rules.CSharp.TokenTypeAnalyzer
445.228 49 SonarAnalyzer.Rules.CSharp.SymbolReferenceAnalyzer
.....
The is be no private info there, just generic times.
Something I noticed also (not sure if relevant) is that there are apparently only 4 analysis done, even though the solution builds about 10 c# projects (not counting the tests projects).
2020-03-02T14:24:19.9399353Z Total analyzer execution time: 3.032 seconds.
2020-03-02T14:24:33.4575050Z Total analyzer execution time: 17.088 seconds.
2020-03-02T14:25:21.5348470Z Total analyzer execution time: 68.835 seconds.
2020-03-02T14:26:11.2910615Z Total analyzer execution time: 64.920 seconds.
3.032 + 17.088 + 68.835 + 64.920 = 153.875 which is under 3 minutes, for those 4 projects.
In these 4 projects, there isn’t an obvious analysis outlier. You could disable S3949 (the rule is CbdeHandlerRule) which takes 7 seconds in one of the project, but you won’t gain a lot.
Regarding the overall performance problem: in the following months we will gradually work to improve the performance of our analysis. You can see some initial baby steps we did with #3168 and #3167.
That’s weird. We’d need access to the full logs in order to figure out what’s happening there. You can send them in a private message.
Hi, since you confirmed it was odd that I was only getting four analysis I decided to re-run the whole process before sending you any more details, to be on the safe side. I reran the build ensuring it would rebuild and I also downloaded the whole logs zip (instead of viewing the raw log in the browser). I now found the other analysis… see Analizers timelog - 2.txt (534.2 KB). Here’s a summary of what I found
The first 4 logs in the new file seem to be the same old four because total times are roughly the same.
There are 10 other analysis (for a total of 14), out of which 8 add up to only 53 seconds
There are then 2 that take 223s (3.7m) and 1,022s (17m)
The one that takes 17m has this line which caught my attention (roughly 10m on a single rule)
My question now is, if I fix all of the instances of the issue, would that decrease the 10 minutes? Or would the analyzer still take all that time? I’m trying to understand my options as we’d rather do the work needed rather than disabling the rule.
Unfortunately fixing reported issues in the code will not improve the performance. The time consuming analysis will have to be done anyway to find out that the candidate issues are real or not.
Performance of UnusedPrivateMember (rules S1144 and S4487) was already identified as pain point in our measurements and we’ll try to address this, as Andrei mentioned.
Whats the update on this? Issue was logged in March and we are also experiencing the same problem
Total analyzer execution time: 1906.825 seconds.
NOTE: Elapsed time may be less than analyzer execution time because analyzers can run concurrently.
Time (s) % Analyzer
1904.096 99 SonarAnalyzer.CSharp, Version=8.10.0.0, Culture=neutral, PublicKeyToken=c5b62af9de6d7244
1871.134 98 SonarAnalyzer.Rules.CSharp.UnusedPrivateMember
)