SonarQube 6.7.x is taking 31 hrs to scan 658k lines of code using sonar scanner 3.4. However with sonar qube 5.6.x it use to take only 30 mins


(Harshal Bhavsar) #1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    Sonarqube 7.4, Sonarqube scanner 3.4
  • what are you trying to achieve
    Analyze the code for sonar issues. Jococo Test reports are excluded
  • what have you tried so far to achieve this
    Exported sonarqube 5.6 ruleset and imported to custom profile in 7.4 but no improvement(Even tried SQ 6.6 and 7.5 version but same result)

I tries sonar scanner versions from 2.4 till 3.4 but same results. If I change to previous sonarqube version i.e., 5.6 its faster in 30 mins

Sonar configurations as below

sonar.sources=project/source, project/source
sonar.sourceEncoding=UTF-8
sonar.java.source=8
sonar.java.coveragePlugin=jacoco
sonar.exclusions=**/*.js,**/*.css,**/*.xml,**/*.html,**/*.xsd
sonar.java.binaries=**/classes/**

Please suggest if any settings to be modified in SonarQube to improve performance of scan. We are upgrading to sonarqube 6.6 or above version as it has better issue tracking system (graph like when it was introduced under activity tab of sonarqube for tracking purpose)

After enabling logs I see below exceptions some times but I guess its normal

org.sonar.java.se.ExplodedGraphWalker$MaximumStepsReachedException: reached limit of 16000 steps for method updateSpecialStatusHistory#212 in class MemberActivityManager at org.sonar.java.se.ExplodedGraphWalker.throwMaxSteps(ExplodedGraphWalker.java:281)

Thanks


(Adam GabryĹ›) #3

Please add information about installed plugins on both servers (7.4 and 5.6).


(Harshal Bhavsar) #4

Hi Adam,

There are no plugins added to these servers and they are out of box


(G Ann Campbell) #5

Hi,

You mentioned that you’re using a custom profile. What happens if you use the Sonar way profile? And just to verify, you don’t have the FindBugs plugin loaded, do you?

 
Ann


(Harshal Bhavsar) #6

Hi Ann, There is no FindBugs plugin loaded to sonarqube server.

Thanks
Harshal


(G Ann Campbell) #7

HI Harshal,

Thanks for the verification. I notice this in your initial post:

sonar.sources=project/source, project/source

Why do you have the same thing listed twice?

It may also help to add a duplications exclusion (project Administration > General Settings > Analysis Scope).

Also, it would help if you could turn on debug logging in your analysis so we could see which steps were taking the most time.

 
Ann


(Harshal Bhavsar) #8

Hi Ann,

Thanks for quick response! Due to confidentiality I named it like this. There are two different code sources.
In dubug logs its not clear which step is taking time. All I am interested is only Java file scanning and exclude other files which I did through sonar properties using exclusion

But as per my analysis, its duplicate detection of blocks or code is taking a very long time instead of applying rules during scan

Duplication in general settings flag is false by default.

I did not configure anything for Duplication exclusion under analysis scope. If I set **/*.java then I assume no duplication issues will be calculated which user doesnt want

Thanks
Harshal


(G Ann Campbell) #9

Hi Harshal,

It’s not quite clear to me whether you’ve worked through this or still have questions, so to be on the safe side…

Yes, setting your duplication exclusion to **/*.java should exclude those files from duplication detection. Given your other exclusions, Java files should be the only ones that are relevant here, but it might be simpler to set the duplication exclusion to **/*.*.

And to be clear, the duplication detection’s primary task is just finding duplications so they can be marked in the interface and metrics can be generated on them. Whether or not issues are raised is almost secondary (and dependent on your quality profile).

Hopefully turning off duplication detection via exclusions has solved your problem. If not, feel free to come back with more details.

 
Ann


(Harshal Bhavsar) #10

Hi Ann,

Thanks for quick response :slight_smile: Appreciate your continuous help. Unfortunately issue still stands unresolved :frowning: even if I add duplication exclusion **/*.java. It still takes same time. I may try **/*.* also but as source points to only java files it wont make a difference in my opinion.

Please find some debug logs attachedconsoleText1.txt (1.8 MB)
before I added above exclusion. Please let me know if you feel something suspicious

Thanks
Harshal


(G Ann Campbell) #11

Hi Harshal,

This isn’t the full debug log. Did the log roll, by chance? Are there additional files numbered 2, 3, …?

Also, what’s your version of SonarJava (found in Administration > Marketplace)? And if it’s not 5.10.1, could you upgrade and provide fresh logs? I know this is a commitment on your part, since analysis takes so long, but it would be best to know up front whether the problem has already been solved in the latest version.

 
Ann