Build of master branch omits coverage for a package, but PR build with no changes does not

branches
coverage

(David Karr) #1

I have a mystifying situation with one project build that is somehow omitting the coverage for an entire package, which drops our coverage percentage well below our required quality gate. This is the build for the “master” branch.

I then created a pull request with essentially no changes (removed some commented code in the POM), which uses the same Jenkinsfile (but omits some steps for a pull request build), but runs the SonarQube scan on all the same code. Comparing the scan results between the two, the PR build had the expected coverage percentage, but the master build had the lower number. I confirmed in the detailed measurements that the PR build was not omitting the coverage for that one package.

Note that in the master build, it had been setting “sonar.cpd.exclusions” to exclude the particular package that is missing coverage. Logically, that shouldn’t affect coverage. In the PR build, I tried toggling that exclusion off and back on. Toggling it off raised the detected duplications, but had no effect on the coverage. Toggling it back on again restored the duplications percentage to be the same as the master build, still with no impact on the coverage percentage.

I did a detailed comparison of the Jenkins console log for the two builds. I thought I would see a difference in the “Java {Main,Test} Files AST scan”, but there was no difference. They had the same number of main and test files to scan.

The only other curious difference I saw was that the master build had MORE unit tests detected, but they were actually not unit tests run by Surefire, they were special “pact” tests that run in a different part of the build. The PR build doesn’t run those “pact” tests, so that partially explains why they weren’t in the unit tests list for the PR build. I don’t know why SonarQube thinks these are unit tests, but I also don’t think it’s likely that their presence could LOWER the resulting coverage.


(David Karr) #2

I was able to repeat this same set of symptoms with a different project pair (master build and PR build) in the same enterprise. Different code, but similar configuration. They both are excluding the coverage for the same package that is being excluded from CPD. Between the two projects, the actual FQPN of the excluded package is different, but they both define similar kinds of classes, which are simple pojos with no behavior.


(G Ann Campbell) #3

Hi,

Since you have 2 cases of this, do you think you could get it down to a minimal reproducer?

 
Thx,
Ann


(David Karr) #4

I think I have a lead on this. I don’t know what the root cause is, but I don’t think it’s caused by SonarQube.