Unit tests added still 0% coverage on new code

SonarQube Cloud
, ,
1

I created a PR in my public repository, I have added proper tests and everything. But sonar test says that the lines are not covered. The details are as follows:

  • ALM used: GitHub
  • CI system used GitHub Actions
  • Scanner command used when applicable (private details masked)

dotnet tool install --global dotnet-sonarscanner
dotnet sonarscanner begin /k:“$(PROJECT_NAME)” /o:“twilio” /d:sonar.host.url=https://sonarcloud.io /d:sonar.login=“${SONAR_TOKEN}” /d:sonar.language=“cs” $(SONAR_SOURCES) /d:sonar.cs.opencover.reportsPaths=“test/lcov.net451.opencover.xml”
dotnet build Twilio.sln > buildsonar.log
dotnet test test/Twilio.Test/Twilio.Test.csproj --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=…/lcov
dotnet sonarscanner end /d:sonar.login=“${SONAR_TOKEN}”

Quality Gate failed
Failed conditions
 0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud
  • Steps to reproduce: Check the logs here

It says that the file is not covered but i have already added tests for coverage. Can someone look into this? Why is this failing? Thanks!

2

Hello @tiwarishubham635

It looks like you are using coverlets ms build integration. According to the documentation, you have to add the NuGet Gallery | coverlet.msbuild 6.0.2 package to your test project.

Can you please add it here?

Please try running

dotnet test test/Twilio.Test/Twilio.Test.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover

locally and check if a coverage report is generated. It should produce output like

...
  [coverlet]
  Calculating coverage result...
   Generating report '... \coverage.opencover.xml'
...
3

Hi! Thanks for taking out your time. I am not a 100% sure if I am following your steps correctly. So I need to cd into folder test/Twilio.test and then run the command

dotnet add package coverlet.msbuild --version 6.0.2

and then run the dotnet test command from root directory. Is that what you are suggesting? I am facing some error on my local. Maybe it’s because of dotnet version but anyways, just wanted to check I am following the right steps?

Thanks!

4

Yes. This is right. The twilio/twilio-csharp/blob/main/test/Twilio.Test/Twilio.Test.csproj should have a new <PackageReference> entry after the change that looks about like so:

  <ItemGroup>
    <PackageReference Include="Microsoft.TestPlatform" Version="17.0.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.8" />
    <PackageReference Include="NSubstitute" Version="4.2.2" />
    <PackageReference Include="NUnitLite" Version="3.13.2" />

    <PackageReference Include="coverlet.msbuild" Version="6.0.2">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
  </ItemGroup>

You should try to make coverage work locally before pushing the changes to your CI pipeline. Getting code coverage files is a bit complicated in .Net, and it is important to make sure the coverage files are created.
The next step is ensuring that the files are found and understood by the scanner for dotnet on your pipeline.

5

Okay so I am able to get the report locally now. What should be the next step? How should I make sure they are being understood by the dotnet scanner?

6

@Martin_Strecker can you help please?

7

@Martin_Strecker I am able to generate a local file lcov.opencover.xml but how do I make sure that it is correctly generated in GitHub as well?

8

Hello @tiwarishubham635,

but how do I make sure that it is correctly generated in GitHub as well?

Please check the following on your pipeline logs

  • Make sure the coverage file is created in the dotnet test test/Twilio.Test/Twilio.Test.csproj part of the script. Note the location (file and foldername) of the coverage file.
  • Make sure the /d:sonar.cs.opencover.reportsPaths command line argument points to the file
  • In the end step, look for “coverage” in the logs and make sure the coverage file is found by the end step
9

I found one thing, in my dotnet sonarscanner begin command there is this part:

/d:sonar.cs.opencover.reportsPaths=“test/lcov.net451.opencover.xml”

But after the command:

dotnet test test/Twilio.Test/Twilio.Test.csproj --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=…/lcov

The output log is like this:

[coverlet] _mapping file name: 'CoverletSourceRootsMapping_Twilio.Test'
  [coverlet] 
  Calculating coverage result...
   Generating report '/home/runner/work/twilio-csharp/twilio-csharp/test/lcov.opencover.xml'

+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+
| Twilio | 0%   | 0%     | 0%     |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+

dotnet sonarscanner end /d:sonar.token="***"

Is the issue coming due to difference in paths test/lcov.opencover.xml and test/lcov.net451.opencover.xml?
If yes, How can I fix it?

Thanks for helping!

10

You need to change your script so that /p:CoverletOutput=... argument and the /d:sonar.cs.opencover.reportsPaths=.. argument point to the same file.
The /p:CoverletOutput seems to be relative to the csproj file: test/Twilio.Test/Twilio.Test.csproj and /p:CoverletOutput=…/lcov results in test/lcov.opencover.xml
/d:sonar.cs.opencover.reportsPaths= should probably be something like /d:sonar.cs.opencover.reportsPaths="test/lcov.opencover.xml"

11

Okay i have some progress. Once I have made the paths same, the code coverage is still 0% but now it is showing 22 lines uncovered while earlier it was showing 8 lines uncovered. I think I just need to figure out how exactly should I write tests to cover them?

Here is the test coverage report - SonarCloud

12

@Martin_Strecker any help would be appreciated :pray:
Thanks for your help throughout

13

I also seeing this - "Code Scanning results may be out of date" security warning in Github, can this be an issue?

14

Can someone update on this?

15

According to the logs of the PR, the coverage report gets imported to SonarCloud:

The coverage summary looks suspicious, though:

It says “0%” coverage for the Twillo module. Please check locally if the coverage reported by coverlet is sensible. Please also follow the documentation for coverlet for trouble shooting.