Code coverage import only works sometimes

Template for a good new topic, formatted with Markdown:

  • ALM used: Azure DevOps
  • CI system used: Azure DevOps
  • Code Coverage tool used: Coco
  • Scanner command used when applicable (private details masked):
    • sonar.cfamily.compile-commands=$(BUILD_WRAPPER_OUTPUTDIR)/compile_commands.json
    • sonar.cfamily.cobertura.reportPaths=$(System.DefaultWorkingDirectory)/coverage_all_unittests.xml

We have setup our project do Code Coverage with Coco.
This produces an xml file which we import in the SonarCloud run.

In SonarCube Cloud, this sometimes gives us the coverage information, but the next builds do not, coverage (total) is at zero again.
The SonarCloudAnalyze has the same output in both cases, also the XML files are the same, but the results just don’t seem to be imported in SonarQube Cloud.
Other (smaller) projects seem to be handled fine and coverage information in being presented (all using the same yml template for producing code coverage xml and doing the SonarQube tasks)

The log of the analyze if very limited, but the same in both cases:
Sensor CFamily [cpp] (done) | time=233973ms

    02:14:20.878 INFO  ------------- Run sensors on project
    02:14:21.006 INFO  Sensor cobertura [cpp]
    02:14:21.116 INFO  Parsing C:\a\2\s\coverage_all_unittests.xml
    02:14:21.545 INFO  Sensor cobertura [cpp] (done) | time=539ms
    02:14:21.545 INFO  Sensor Zero Coverage Sensor
    02:14:22.011 INFO  Sensor Zero Coverage Sensor (done) | time=465ms

Is there a way to get more information on why the results are not shown in SonarQube?

Thank you
Patrick Riphagen

Hey @PPRiphagen!

It would be really weird for the coverage to appear/disappear without anything else being different in the background.

More logging info is available at DEBUG level (sonar.verbose=true).

Probably the best debugging information would be:

  • DEBUG level analysis logs from two runs – one where coverage data appears, and a successive one where the coverage is absent
  • A copy of the coverage report in each case

Hi Colin

Thank you very much, looking forward to the results with this option.
We have not had many builds yet (it’s a big build, taking couple of hours). Until now the 1st 2 builds did produce results, last 2 did not.
I just added the option and will check the results. Hopefully I can get more information now that the verbose level has been changed.

Will update once I have more information
Thank you

Using the DEBUG level, I think I discoved why the coverage is sometimes zero.

We run the Code Coverage and the SonarCloud analysis in 2 different jobs, on 2 different agents.
Since the xml contains the full path to the folder, the folder from the CodeCoverage agent can be /a/2/s, where the analyze step is running in C:/a/1/s, causing the relevant files cannot be found.

I did try removing some file path elements using cmedit, but so far that does not seem to get me to the right folder depth, then it cannot resolve any file.

The xml also contains the <package name=“/a/2/s/project_name” and filename=“/a/2/sproject_name”

In the SonarCloudPrepare task, we set the sources parameter:
cliSources: project_name/src

I don’t know how to set these paths to be relative?

Thank you

Patrick

Hey @PPRiphagen

I’ll admit this is an area where I’m a little rusty. However, I believe the key is that the files referenced in your Cobertura report (specifically the attribute filename, everything else is ignored when resolving the file) need to be relative to the value of sonar.projectBaseDir.

You can find out how this value is currently set (probably by default) by looking in your analysis logs:

16:03:48.451 INFO: Base dir: /Users/colin/source/coberturatest/src

So given the following directory structure:

colin@MAC-L0251 coberturatest % tree
.
├── cobertura.xml
└── src
    ├── file1.cpp
    ├── file2.cpp
    └── lib.cpp

2 directories, 4 files

My coverage report needs to look like this:

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE coverage SYSTEM 'http://cobertura.sourceforge.net/xml/coverage-04.dtd'>
<coverage line-rate="0.9333333333333333" branch-rate="0.75" lines-covered="14" lines-valid="15"
          branches-covered="3" branches-valid="4" complexity="0.0" timestamp="1688569050"
          version="gcovr 5.0">
    <sources>
        <source>.</source>
    </sources>
    <packages>
        <package name="src" line-rate="0.9333333333333333" branch-rate="0.75" complexity="0.0">
            <classes>
                <class name="lib_cpp" filename="lib.cpp" line-rate="0.875" branch-rate="0.75"
                       complexity="0.0">
                    <methods/>
                    <lines>
                        <line number="2" hits="4" branch="false"/>
                        <line number="3" hits="4" branch="false"/>
                        <line number="5" hits="4" branch="true" condition-coverage="100% (2/2)">
                            <conditions>
                                <condition number="0" type="jump" coverage="100%"/>
                            </conditions>
                        </line>
                        <line number="6" hits="2" branch="false"/>
                        <line number="8" hits="2" branch="false"/>
                        <line number="14" hits="4" branch="true" condition-coverage="50% (1/2)">
                            <conditions>
                                <condition number="0" type="jump" coverage="50%"/>
                            </conditions>
                        </line>
                        <line number="15" hits="0" branch="false"/>
                        <line number="17" hits="4" branch="false"/>
                    </lines>
                </class>
                <class name="test1_cpp" filename="file1.cpp" line-rate="1.0" branch-rate="0.0"
                       complexity="0.0">
                    <methods/>
                    <lines>
                        <line number="3" hits="4" branch="false"/>
                        <line number="4" hits="2" branch="false"/>
                        <line number="5" hits="2" branch="false"/>
                    </lines>
                </class>
                <class name="test2_cpp" filename="file2.cpp" line-rate="1.0" branch-rate="0.0"
                       complexity="0.0">
                    <methods/>
                    <lines>
                        <line number="3" hits="2" branch="false"/>
                        <line number="4" hits="2" branch="false"/>
                        <line number="7" hits="2" branch="false"/>
                        <line number="8" hits="2" branch="false"/>
                    </lines>
                </class>
            </classes>
        </package>
    </packages>
</coverage>

And coverage is successfully imported.

If my base directory is one higher (/Users/colin/source/coberturatest/), /src/ must be added to all those filenames in the report.

Let me know if this helps you get any closer.

Hi Colin,

I will experiment some more, reading up at the parameter you mention, I just read that the latest version does support that anymore (SonarCloudPrepare@3.0.3 no longer respects sonar.projectBaseDir?)

But I will see if I can modify the paths using the now default Devops source directory.

Will let you know about the results

For what it’s worth, that is specific to using the Scanner for .NET (scannerMode: 'dotnet' rather than scannerMode: 'cli'). Which one are you using?

Interesting, we use ‘cli’. I will give it a try today, thank you