Extermely slow scanner

We are using Enterprise 9.9 together with Team City and the Scanner plugin. I can currently only use version 4.8 of the scanner.

Scanning our large .net framework application (120 projects) can take 20 + minutes, even with a minimal quality profile (I just selected blocker rules, and deactivated all js and typescript rules). This would make our PR build take over 40 minutes.

Ive also tried building a single project. I dont think I can use the dotnet tool version of the scanner.

It seems like nearly all of the time is spent here:

Reading UCFGs from: C:\buildAgent\work\af74023734cabac3\.sonarqube\out\ucfg_cs2

Is there anything else I can try? If not I’ll need to do some sort of overnight scan and alert developers.

I have an 18mb verbose log file if someone wants to see it :confused:

Do not share screenshots of logs – share the text itself (bonus points for being well-formatted)!

Hi,

Which scanner are you using? SonarScanner CLI, or SonarScanner for .NET?

And why can you only use the 4.8 version? For the SonarScanner for .NET, v4.8 is incredibly old (Nov. 2019). It’s version 5.9 of the scanner (note that the current version is 6.1) that adds support for fast PR analysis. So the fact that your PRs are currently slow - even with SonarQube 9.9, which supports fast PR analysis - is not surprising.

 
Ann

Thats good to know - if it’s doing a full full analysis of 120 projects… that could take a while.

There seems to be many ways to run the scanner in our company. The project Im on is about 15 years old so Im just working with the original build scripts and our Team City installation only seems to work on this version. There is a v 6 listed but it wont work. Some java version error.
Ive got a couple more things to try but I’ll see if I can get the TC team to update the agents.

Thanks!!

Hi,

Yeah… You’ll want to upgrade to a current version of Java on that build agent.

 
HTH,
Ann

Yeah, trying to convince the team…
Otherwise I need to install team city locally, configure the git connections, learn kotlin, pray to some god etc…

I hate legacy apps :>

1 Like

Hi. I tried a different method to run the scanner v 5.5 and it’s still taking around 20 minutes. It the cli version.

I have limited access to the team city platform so I dont have any choice currently - unless I redo the whole build pipeline using a container. Im happy to do this but Im not convinced it will be any better.

Hi,

Has the underlying branch been analyzed? If PR analysis doesn’t have access to a good cache from the underlying branch, then it’ll still have to do a full analysis.

Can you share your analysis log?

 
Ann

I can share the log but not publically. Can I do that here?

Id say that will be the issue. Im not sure where the cache could be stored - we would need to persist it off the build agent I assume since the agents are randomly assigned to tasks. I’ll do some research.

Hi,

The cache is stored server-side.

 
Ann

Oh. In that case, I would need to do a full analysis on master. The last one was done about 2 years ago when they disabled it.

1 Like

So I now have a successful build and analysis of our master branch and pull request branches. Its still very slow though. For the New Code settings I have master set to Previous Version and Project reference branch set to master. Is that correct?

Im also seeing these warnings:

  • There are problems with file encoding in the source code. Please check the scanner logs for more details.
  • Decoration failed. No revision information available
  • Code scanning alerts upload failed. No revision or analysis results available

Hi,

Thanks for following up with the warnings.

This one is unrelated, but you may want to follow up on it later

This is about SCM information. It looks like analysis didn’t have all the SCM data it needed to correctly identify what’s in the PR. You should find more evidence of that in your analysis log, if you look.

Without that data - without being able to identify what changed in the PR - we’re still back at analyzing everything.

 
Ann

Looking in the log I can see this:

[21:02:09] :	 [Step 6/6] AB40ADC2-B347-4304-A8A6-5F85BCA412CC.sonar.working.directory=C:\\buildAgent\\work\\2aa7a631290d49d9\\.sonarqube\\out\\.sonar\\mod121
[21:02:09] :	 [Step 6/6] project.home=.
[21:02:09] :	 [Step 6/6] sonar.host.url=http://sonar....
[21:02:09] :	 [Step 6/6] sonar.sources=src
[21:02:09] :	 [Step 6/6] sonar.pullrequest.branch=master
[21:02:09] :	 [Step 6/6] sonar.pullrequest.key=43735
[21:02:09] :	 [Step 6/6] sonar.pullrequest.base=feature/blahblah
[21:02:09] :	 [Step 6/6] sonar.pullrequest.provider=github
[21:02:09] :	 [Step 6/6] sonar.pullrequest.github.repository=blah/repo
[21:02:09] :	 [Step 6/6] sonar.visualstudio.enable=false
[21:02:09] :	 [Step 6/6] 
[21:02:09] :	 [Step 6/6] sonar.modules=2293160B-8ED1-45D9-8773-2B013C4E0A57,F37337B8-40DE-4ED5-8D65-6400F2B68495 ... very long line

then:

|[21:05:10] :| [Step 6/6] 08:05:10.459 INFO: Sensor PHP sensor [php]
|[21:05:10] :| [Step 6/6] 08:05:10.461 WARN: The sonar.modules is a deprecated property and should not be used anymore, it inhibits an optimized analysis
|[21:05:10] :| [Step 6/6] 08:05:10.474 INFO: Starting PHP symbol indexer

We dont use PHP and I excluded any rules for the language in the profile.

Im not sure where this is coming from. There’s nothing in our pipeline parameters for it. Maybe it’s not relevant?

Hi,

Each sensor present will be invoked.

If you want to upgrade to 10.4 (10.4.1 due today, so wait til that’s out) then you’ll be able to enable a mechanism to download (and run) only the language analyzers relevant for the current project. Note the caveat.

 
HTH,
Ann

Hi again,

I should have mentioned that this feature will only save a few minutes at best. If you check the timing on the irrelevant sensor invocations, you’ll see they’re sub-second.

 
Ann

I’ll let the team know about the upgrade, unlikely that will happen soon though.

So is the sonar.modules setting anything to worry about?

Hi,

I’m guessing you didn’t manually set sonar.modules, right? If so, you can just ignore that warning.

 
Ann

Correct. Thanks. Will keep checking the log… 100k + lines :confused:

1 Like