Getting issues After upgrading to Sonarqube Developer edition version 8.9.9 from sonarqube version 7

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    Old version community edition 7.1
    New Version Developer edition 8.9.9

  • what are you trying to achieve
    We are migrating from Version 7.1 to 8.9.9 developer edition - we are running scans against same
    repository for both versions but we are seeing inconsistent results

    We have observed that all of our unit test cases are not being picked up by new version 8.9.9. On
    version 7.1, we can see that there are 114 unit tests, but in version 8.9.9 sonarqube is only displaying
    14 unit tests. We are using the sonarqube gradle plugin and using jacocoMergeReport to create the
    xml file from the exec file (requirement for sonarqube 8.2+)

    • We are also seeing higher code coverage results being produced with new version 8.9.9 than we are
      seeing in version 7.1 (by about 8%)
  • what have you tried so far to achieve this
    We performed same build process against the microservice and upload results to new version 8.9.9 as
    same as version 7.1

Do not share screenshots of logs – share the text itself (bonus points for being well-formatted)!

Hi,

Welcome to the community!

Is this about test execution or test coverage? They’re two different topics, and we prefer to keep it to one topic per thread.

That said, SonarQube goes by the reports you feed into analysis. The first step would be to verify that the data you were providing to the SonarQube 7.1 analysis is the same as what you’re feeding in to the 8.9 analysis.

 
Ann

Hello,

I am a coworker of the original poster, and I’ll try to elaborate. This current request has to deal with inconsistencies between the displays of code coverage reports that we ship to SonarQube from our Jenkins pipelines. We’re currently leveraging the sonarqube gradle task as well as a jacocoMergeReport when shipping to SonarQube v8.9.9 to build the .xml. file required as versions above 8.2 dont accept the .exec binary anymore.

Example that we are seeing:

A)
Jenkins pipeline → Runs unit tests → produces .exec → SonarQube v7.1
Result: Overall coverage 94.4% over 115 unit tests

B)
Jenkins pipeline → Runs unit tests → produces .exec → jacocoMergeReport creates XML from exec → SonarQube v8.9.9
Result: Overall coverage 96.7% over 14 unit tests

On version 8.9.9, the unit tests I see showing in SonarQube all come from one directory:
[our-microservice/src/test/java/com/ourcompany/microservice/example/directory/here/] whereas in version 7.1, we see unit tests from that directory as well as other directories under [our-microservice/src/groovy/etc/etc]. It seems as if those tests that live under the src/groovy directory aren’t making displaying in the newer version, but both src/java and src/groovy show in the older version. The java tests have a .java file extension, while the groovy unit tests have a .groovy extension. I’m wondering if that might be playing a role here.

Thanks!

HI @Jason_Ritter,

Welcome to the community!

This is still two different topics: coverage and unit test execution. :smiley:

While these numbers seem related, to SonarQube the relationship is only tangential. SonarQube goes by the reports you feed it for both of these. It does not calculate one from the other.

Let’s focus on coverage.

  • Have you verified that the reports you’ve been feeding into SQ 7.1 reflect *exactly *the same data as the reports you’re feeding into SQ 8.9? File for file, line for line, condition for condition?
  • Did your count of lines_to_cover (you’ll find this listed with the Coverage metrics) stay the same after the migration? Because that’s (most of) the denominator when overall coverage% is calculated
  • Did your sonar.sources definition stay exactly the same?
  • Did exclusions change?
  • Do your analysis logs before & after the migration show the same number of files to analyze?

 
Ann

Thanks for the quick response!

Have you verified that the reports you’ve been feeding into SQ 7.1 reflect *exactly *the same data as the reports you’re feeding into SQ 8.9? File for file, line for line, condition for condition?

We can try and do a more thorough eval of the two files. I’ll have to get back to you on this one.

Did your count of lines_to_cover (you’ll find this listed with the Coverage metrics) stay the same after the migration? Because that’s (most of) the denominator when overall coverage% is calculated

No. On 7.1 I see 650 overall lines for this project, and 648 on the new 8.9.9 version. The rest of the overall numbers are pretty close to each other, similar to the overall lines. The scans were done close in time to one another, but it’s possible that the more recent scan on 7.1 just had some new code added to explain the minor differences.

Did your sonar.sources definition stay exactly the same?

We don’t explicitly define sonar.sources. Our main source files exist under /microservicename/src/main, but those main files are being picked up and I can see them listed when I click Overall -> Coverage.

Did exclusions change?

No

Do your analysis logs before & after the migration show the same number of files to analyze?

Will check once I grab admin privileges on the new instance.

I did actually just find this StackOverflow page that describes the issue I uncovered while responding to this post. I do see that we have the Groovy plugin installed on our old version, but I’m going to need some extra access to look at our new instance to see if we have the same plugin (sounds like we might not based off of that SO post!)

Hi,

I know two lines doesn’t seem like much, but neither does the 2.3% difference you’re investigating. Probably also worth comparing the number of conditions since, per the docs I linked above

Coverage = (CT + CF + LC)/(2*B + EL)
where:

  • B = total number of conditions
  • EL = total number of executable lines (lines_to_cover)

BTW, what else got upgraded in this? JaCoCo?

 
Ann

number of conditions

Old:
Conditions to cover: 80
Uncovered conditions: 10
Condition coverage: 87.5%

New:
Conditions to cover: 80
Uncovered conditions: 5
Condition coverage: 93.8%

BTW, what else got upgraded in this? JaCoCo?

For now my understanding is that only the SonarQube version was upgraded. We’re still using the same version of the JaCoCo plugin that we were using prior to the migration. I’ll let shivaprasadM fill in any additional details however.

Hi,

Your condition coverage shift is enough - on a 648-650 LOC project that (without all the numbers to double-check the math) I guess it explains the 2.3% difference in overall coverage.

At this point, I think you need to dig into the coverage reports you’re feeding in to analysis. At a guess, the change is happening when you merge the reports.

 
HTH,
Ann

A post was split to a new topic: Plugin error on upgrade