[Coverage & Test Data] Generate Reports for Apex, C/C++, Objective-C, Go, JS/TS and Python

Hello,

Here is a quick guide on how to generate coverage and test data reports for several languages. For other languages or to know how to integrate these reports into SonarQube or SonarCloud, have a look at Test Coverage & Execution | SonarQube Docs.

These commands were used when we implemented this feature. If you are using other commands feel free to share them as replies.
If you have trouble importing these reports, please ask for help on the dedicated "Get help" category.

Apex

Coverage

Execute the unit tests and generate the coverage report. This can be done using the following command:

sfdx force:apex: test :run -c -d <output_directory> -r json --wait <timeout_minutes>

Example:

sfdx force:apex: test :run -c -d . -r json --wait 4

This will generate a file test-result-codecoverage.json in the provided directory.

C/C++/Objective-C

Coverage

Visual Studio Reports

Run the tests (Debug\UnitTest1.dll in our example) enabling coverage, this is going to create a TestResult directory.

Example:

C:\path\to\vstest.console.exe Debug\UnitTest1.dll /EnableCodeCoverage

Create code coverage XML report (codeCoverage.xml in our case) using codecoverage.exe (available in the Visual Studio installation folder under Team Tools\Dynamic Code Coverage Tools directory).

Example:

C:\path\to\codecoverage.exe analyze /output:codeCoverage.xml TestResult\<RESULT_FOLDER>\<REPORT_NAME>.coverage

JavaScript/TypeScript

Test Data

Converters to Generic Test Data format

Python

Coverage

Tool: Coverage Tool provided by Ned Batchelder

Commands:

coverage erase
coverage run --branch --source=<python packages> <program> 
coverage xml -i

Make sure to put all packages to measure into the --source parameter. That ensures that coverage will report zero coverage on all untouched files, as you most probably want to. To make this work, make sure to meet two prerequisites:

  • use Coverage Tool 3.4+
  • the packages should be packages, i.e. every directory on the way to the sources should contain an init.py file.

Test Data

Tool: Nose or pytest.
Support Format: XUnit

Command with Nose:

nosetests --with-xunit [tests to execute]

To get a more accurate execution time statistics make sure to use Nose 1.1.0+

By default, the nosetests command will generate the report nosetests.xml into a directory named nosetests created where the command is executed.

Command with pytest:

pytest --junitxml=pytest-report.xml

See the official documentation for more information on how to configure and run tests with pytest.

Go

Coverage

Tool Command
Go Coverage Tool go test -coverprofile=coverage.out

Test Data

Tool Command
“go test” go test -json > report.json

This content was previously part of SonarQube’s documentation but it applies for both SonarQube and SonarCloud so we though it would be better to share it here.

2 Likes

A post was merged into an existing topic: Why there is difference in codecoverage

This is confusing:

It should be coverage xml -i (no .)

And it is necessary to use -Dsonar.python.coverage.reportPaths=./coverage.xml to pull in that xml.

Fixed. Thanks @jayvdb

As this topic is now the semi-official docs for these languages, I think it would be helpful to list the sonar property which needs to be set for each language. e.g. for JavaScript, with GitHub - tornaia/karma-sonarqube-unit-reporter: A Karma plugin. Unit tests execution report for SonarQube/Generic Coverage Plugin , sonar.javascript.lcov.reportPaths=path/to/coverage/lcov.info is needed.

1 Like

A post was split to a new topic: Apex coverage number doesn’t match Salesforce’s numbers