Apex Code-Coverage: Sonar values differ from Salesforce

To whom it may concern:

We’re running static code analysis on two salesforce projects (apex code) through Azure pipelines. Sonar Cube publishes the code coverage results as being 47% for one project, and 50% for the other.

Salesforce has a quality gate which states that “unit tests must cover at least 75% of your Apex code” for code to be published. Obviously we’re intrigued.

Analyzing the resulting test-result-codecoverage.json file and calculating the code coverage (using 100 values for nulls), we get an average of 84% for the both projects.

Can you please assist in getting this right? Thank you!

  • ALM used:
    Azure DevOps
  • CI system used:
    Azure DevOps
  • Command used for obtaining code coverage:
    sfdx force:apex:test:run -c -r json -l RunLocalTests -w 60 -d $(test-result-localFolder) -u $(CodeCoverageUser)
  • Advanced->Addditional Properties passed to “Prepare analysis on SonarCloud” task:
    sonar.exclusions=**/*.xml, **/*__*.cls
    sonar.apex.coverage.reportPath=$(test-result-codecoverage-file)
  • Language
    Apex
  • Error observed
    Different code coverage result
  • Steps to reproduce
    In our case this happens with both our salesforce projects

I’m guessing no one else has quality gates for Salesforce code coverage.

Hello Susana,

While browsing uncovered code in your SonarCloud project, are you sure that there is only Apex code? Or is there any other language that has less coverage that reduces the overall coverage of your project?

test-result-codecoverage.json should look like this:
Selection_081

It both provides file names and lines that are covered. If you compare test-result-codecoverage.json and SonarQube project un covered lines, do you manage to identified differences?
If yes could you share a screenshot that shows the difference between your report and the SonarQube project?

JD.

Hi Jean-Denis,

Thank you for following up on this.

Only .cls files :slight_smile:
Yup, our test-result-codecoverage.json looks just like that.
The differences I find between Salesforce and/or test-result-codecoverage.json and the SonarCloud results are that some classes are not mentioned in the result file (1st image), Salesforce reports them with 100% coverage (2nd image) and SonarCloud presents them with 0% coverage (3rd image):
image image

Again, thank you.
Susana

Hello Susana,

The problem is that to make your point you used different classes which makes the problem difficult to confirm. You provided coverage for:

  • AccountTriggerHandler coverage in json file and in SalesForce
  • addDestinationPopUp_Ctrl in SalesForce and SonarQube

What is the coverage of addDestinationPopUp_Ctrl in the Json file?
What is the coverage of AccountTriggerHandler in SonarQube?

Note that if there are differences, it would be nice to see on which lines to confirm that these lines are considered as lines to cover both in SonarQube and SalesForce.

Best regards,
JD.

Hi Jean-Denis,

Thank you for following up on this.

Sorry if I wasn’t clear; throughout my reply I was always referring to addDestinationPopUp_Ctrl:
addDestinationPopUp_Ctrl has no entry in the .json file
image
addDestinationPopUp_Ctrl has 100% of coverage in Salesforce
image
addDestinationPopUp_Ctrl has 0% of coverage in Sonar

AccountTriggerHandler is not an issue: this class is being correctly handled by Sonar: 93 out of 95 lines covered, which gives a 97-98% coverage.
.json file:
image
Salesforce:
image
Sonar:

Again, thank you!

It means what is in SonarCloud matches your Json file. This is what is expected and I guess there is no problem on SonarCloud side. I invite you to understand why sfdx report is not matching your SalesForce coverage results.

I may have an idea about this. If I am correct, sfdx retrieves the results of a test execution. Are you sure that you are running your full test campaign in one shot? If not, Be aware that SalesForce may aggregates results of many tests runs which could explain the mismatch between SonarCloud and SalesForce.

Good luck and when you will fine, I will be happy to have the final word on this. :wink:

JD.

We actually have the same issue, sonar is reporting 0% coverage on .cls test files and all other files not included in the SFDX report. @SuChuDom did you manage to solve it?