SQ 8.5 - c# .NET 4.8 scans getting stuck

bkavanaugh · October 12, 2020, 8:28pm

versions used SonarQube 8.5
scan from Azure DevOps build pipeline using msbuild scanner 4.11.0

Since upgrading to 8.5 this morning, two of our large C# API projects, which are .NET 4.8, are getting stuck. The task says it is still running, but the logs stop capturing anything until I kill the task.

In the logs, they are all ending like this:

2020-10-12T16:02:07.3728074Z 11:02:07.371 INFO: All rules entrypoints : 537 Retained UCFGs : 14156
2020-10-12T16:02:08.2669892Z 11:02:08.265 INFO: rule: S5131, entrypoints: 15
2020-10-12T16:02:08.2672192Z 11:02:08.266 DEBUG: Running rule roslyn.sonaranalyzer.security.cs:S5131
2020-10-12T16:02:08.2672810Z 11:02:08.266 INFO: Running symbolic analysis
2020-10-12T16:02:08.2707454Z 11:02:08.269 DEBUG: loaded 73 sanitizers for rule S5131
2020-10-12T16:02:08.2730006Z 11:02:08.272 DEBUG: Resource file roslyn.sonaranalyzer.security.cs/passthroughs/S5131.json was not read
2020-10-12T16:02:08.2731274Z 11:02:08.272 DEBUG: loaded 171 passthroughs for rule S5131
2020-10-12T16:02:08.2734260Z 11:02:08.272 DEBUG: Resource file roslyn.sonaranalyzer.security.cs/collectionHandlers/common.json was not read
2020-10-12T16:02:08.2735244Z 11:02:08.272 DEBUG: Resource file roslyn.sonaranalyzer.security.cs/collectionHandlers/S5131.json was not read
2020-10-12T16:02:08.2736333Z 11:02:08.272 DEBUG: loaded 0 collectionHandlers for rule S5131
2020-10-12T16:02:08.4579526Z 11:02:08.456 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argEntity .
2020-10-12T16:02:08.4580611Z 11:02:08.457 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argEntity .
2020-10-12T16:02:08.4581361Z 11:02:08.457 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argAlClientId .
2020-10-12T16:02:08.4658129Z 11:02:08.464 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argEntity .
2020-10-12T16:02:08.4661218Z 11:02:08.465 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argUserKey .
2020-10-12T16:02:08.4666526Z 11:02:08.465 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argUserKey .
2020-10-12T16:02:08.4676962Z 11:02:08.466 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argPersonal .
2020-10-12T16:02:08.4678449Z 11:02:08.467 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argLoginUserId .
2020-10-12T16:02:08.4682987Z 11:02:08.467 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argUserGuid .
2020-10-12T16:02:08.4684422Z 11:02:08.467 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argPersonal .
2020-10-12T16:02:08.4933116Z 11:02:08.492 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argId .
2020-10-12T16:02:08.4934015Z 11:02:08.492 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argDefaultRefreshTokenLifetime .
2020-10-12T16:02:08.4955193Z 11:02:08.494 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argDefaultRefreshTokenLifetime .
2020-10-12T16:02:08.5046792Z 11:02:08.503 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argExternalId .
2020-10-12T16:02:08.5047389Z 11:02:08.504 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with argExternalId .
2020-10-12T16:02:08.5088152Z 11:02:08.507 DEBUG: Did not expect to visit symbol class com.sonar.security.analysis.D.A.N with arghooks .
2020-10-12T20:00:06.2548610Z ##[error]The operation was canceled.
2020-10-12T20:00:06.2556021Z ##[section]Finishing: Run Code Analysis

Edit: I found this post in GitHub which references the same rule. https://github.com/SonarSource/sonar-scanner-msbuild/issues/827. However, I’ve searched the logs generated for some of the key words listed there (ProjectCapability, SonarQubeTestProject, test project) and do not get any hits. These are projects that don’t change very often, and were working prior to updating from 8.4.2 to 8.5.0 yesterday morning.

jbeleites · October 14, 2020, 8:42am

Hello Brian, thanks for the report!

We are aware of a recent bug that occurs in certain edge conditions when analyzing dotnet projects. The result is a non-terminating loop, your problem looks a lot like it. Could you check whether you are experiencing high (and rapidly increasing) memory usage when your scan gets stuck?

bkavanaugh · October 14, 2020, 12:47pm

I’m not sure which process I should monitor, but here’s what I saw on our build servers for the two projects.

Legacy Project
Java SE binary: was ~1900mb usage, spiked briefly to 1941mb, then dropped back to 1892. Stayed at about 13.5% CPU usage after that until I canceled the task.
MsBuild.Scanner: stayed stable at 123mb usage

API Project
Java SE binary: was ~1885mb usage, spiked briefly to 1929mb, then dropped back to 1885. Stayed at about 13.5% CPU usage after that until I canceled the task.
MSBuild.Scanner: stayed stable at 33mb usage

jbeleites · October 20, 2020, 1:32pm

It could be the case that your Java threads have a maximum of ~2GB memory available, hence it is staying at around that without getting any further. You can try disabling rule S5131 for now and see if your scan finishes. (If this also happens on other rules, try disabling them for now as well). We have a fix for the bug mentioned in my previous reply, which will be shipped with the next release of SonarQube, the chances are pretty high that this will fix your problem and you can re-enable the rules at that point.

bkavanaugh · October 20, 2020, 7:00pm

Disabling rules eventually got the scans working again. Here are the ones I had to disable, if that helps:

S5131
S3649
S2076
S2631
S2083
S5144
S5146
S5135
S5145

With the possible exception of one of those, I don’t recall ever seeing them triggered in our scans, but it would be nice to get these re-enabled quickly.

jbeleites · January 5, 2021, 4:12pm

Happy new year! As of SonarQube version 8.6 this problem should now be fixed. If you haven’t tried yet, you may want to consider upgrading and re-enabling the rules in question. Please let us know if you are still experiencing issues despite upgrading!

system · October 27, 2021, 8:36am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.