[Coverage & Test Data] Generate Reports for Apex, C/C++, Objective-C, Go, JS/TS and Python

go
sonarcfamily
coverage
python
apex
(Alexandre Gigleux) #1

Hello,

Here is a quick guide on how to generate coverage and test data reports for several languages. For other languages or to know how to integrate these reports into SonarQube or SonarCloud, have a look at https://docs.sonarqube.org/latest/analysis/coverage/.

These commands were used when we implemented this feature. If you are using other commands feel free to share them as replies.
If you have trouble importing these reports, please ask for help on the dedicated "Get help" category.

Apex

Coverage

Execute the unit tests and generate the coverage report. This can be done using the following command:

sfdx force:apex: test :run -c -d <output_directory> -r json --wait <timeout_minutes>

Example:

sfdx force:apex: test :run -c -d . -r json --wait 4

This will generate a file test-result-codecoverage.json in the provided directory.

C/C++/Objective-C

Coverage

Visual Studio Reports

Run the tests (Debug\UnitTest1.dll in our example) enabling coverage, this is going to create a TestResult directory.

Example:

C:\path\to\vstest.console.exe Debug\UnitTest1.dll /EnableCodeCoverage

Create code coverage XML report (codeCoverage.xml in our case) using codecoverage.exe (available in the Visual Studio installation folder under Team Tools\Dynamic Code Coverage Tools directory).

Example:

C:\path\to\codecoverage.exe analyze /output:codeCoverage.xml TestResult\<RESULT_FOLDER>\<REPORT_NAME>.coverage

JavaScript/TypeScript

Test Data

Converters to Generic Test Data format

Python

Coverage

Tool: Coverage Tool provided by Ned Batchelder

Commands:

coverage erase
coverage run --branch --source=<python packages> <program> 
coverage xml -i.

Make sure to put all packages to measure into the --source parameter. That ensures that coverage will report zero coverage on all untouched files, as you most probably want to. To make this work, make sure to meet two prerequisites:

  • use Coverage Tool 3.4+
  • the packages should be packages, i.e. every directory on the way to the sources should contain an init.py file.

Test Data

Tool: Nose or pytest.
Support Format: XUnit

Command with Nose:

nosetests --with-xunit [tests to execute]

To get a more accurate execution time statistics make sure to use Nose 1.1.0+

By default, the nosetests command will generate the report nosetests.xml into a directory named nosetests created where the command is executed.

Command with pytest:

pytest --junitxml=pytest-report.xml

See the official documentation for more information on how to configure and run tests with pytest.

Go

Coverage

Tool Command
Go Coverage Tool go test -coverprofile=coverage.out

Test Data

Tool Command
“go test” go test -json > report.json

This content was previously part of SonarQube’s documentation but it applies for both SonarQube and SonarCloud so we though it would be better to share it here.