We’re running static code analysis on two salesforce projects (apex code) through Azure pipelines. Sonar Cube publishes the code coverage results as being 47% for one project, and 50% for the other.
Analyzing the resulting test-result-codecoverage.json file and calculating the code coverage (using 100 values for nulls), we get an average of 84% for the both projects.
Can you please assist in getting this right? Thank you!
ALM used:
Azure DevOps
CI system used:
Azure DevOps
Command used for obtaining code coverage: sfdx force:apex:test:run -c -r json -l RunLocalTests -w 60 -d $(test-result-localFolder) -u $(CodeCoverageUser)
Advanced->Addditional Properties passed to “Prepare analysis on SonarCloud” task: sonar.exclusions=**/*.xml, **/*__*.cls sonar.apex.coverage.reportPath=$(test-result-codecoverage-file)
Language
Apex
Error observed
Different code coverage result
Steps to reproduce
In our case this happens with both our salesforce projects
While browsing uncovered code in your SonarCloud project, are you sure that there is only Apex code? Or is there any other language that has less coverage that reduces the overall coverage of your project?
test-result-codecoverage.json should look like this:
It both provides file names and lines that are covered. If you compare test-result-codecoverage.json and SonarQube project un covered lines, do you manage to identified differences?
If yes could you share a screenshot that shows the difference between your report and the SonarQube project?
Only .cls files
Yup, our test-result-codecoverage.json looks just like that.
The differences I find between Salesforce and/or test-result-codecoverage.json and the SonarCloud results are that some classes are not mentioned in the result file (1st image), Salesforce reports them with 100% coverage (2nd image) and SonarCloud presents them with 0% coverage (3rd image):
The problem is that to make your point you used different classes which makes the problem difficult to confirm. You provided coverage for:
AccountTriggerHandler coverage in json file and in SalesForce
addDestinationPopUp_Ctrl in SalesForce and SonarQube
What is the coverage of addDestinationPopUp_Ctrl in the Json file?
What is the coverage of AccountTriggerHandler in SonarQube?
Note that if there are differences, it would be nice to see on which lines to confirm that these lines are considered as lines to cover both in SonarQube and SalesForce.
Sorry if I wasn’t clear; throughout my reply I was always referring to addDestinationPopUp_Ctrl: addDestinationPopUp_Ctrl has no entry in the .json file addDestinationPopUp_Ctrl has 100% of coverage in Salesforce addDestinationPopUp_Ctrl has 0% of coverage in Sonar
AccountTriggerHandler is not an issue: this class is being correctly handled by Sonar: 93 out of 95 lines covered, which gives a 97-98% coverage.
.json file:
Salesforce:
Sonar:
It means what is in SonarCloud matches your Json file. This is what is expected and I guess there is no problem on SonarCloud side. I invite you to understand why sfdx report is not matching your SalesForce coverage results.
I may have an idea about this. If I am correct, sfdx retrieves the results of a test execution. Are you sure that you are running your full test campaign in one shot? If not, Be aware that SalesForce may aggregates results of many tests runs which could explain the mismatch between SonarCloud and SalesForce.
Good luck and when you will fine, I will be happy to have the final word on this.
We actually have the same issue, sonar is reporting 0% coverage on .cls test files and all other files not included in the SFDX report. @SuChuDom did you manage to solve it?
