Code Coverage keeps bouncing between real coverage and 0

[ShaleenAg (Shaleen Agarwal)]

Template for a good new topic, formatted with Markdown:

• ALM used (GitHub, Bitbucket Cloud, Azure DevOps)
  Gitlab
• CI system used (Bitbucket Cloud, Azure DevOps, Travis CI, Circle CI
  Gitlab Runner
• Scanner command used when applicable (private details masked)

  script:
    - sonar-scanner 
        -Dsonar.token=$SONAR_TOKEN_BACKEND
        -Dsonar.projectKey=masked_key
        -Dsonar.organization=masked_name
        -Dsonar.host.url=$SONAR_HOST_URL
        -Dsonar.sources=.
        -Dsonar.branch.name=$CI_COMMIT_REF_NAME
        -Dsonar.python.coverage.reportPaths=coverage.xml

• Languages of the repository
  Python
• Only if the SonarCloud project is public, the URL
  • And if you need help with pull request decoration, then the URL to the PR too - NA
• Error observed (wrap logs/code around with triple quotes ``` for proper formatting) - NA
• Steps to reproduce - NA
• Potential workaround - NA

Do not share screenshots of logs – share the text itself (bonus points for being well-formatted)!

Hi folks, Our test coverage keeps swapping between 0% to the real coverage randomly, at first we figured it was somehow related to when we created PRs since that was when coverage randomly dropped, but even over the weekend, when no code was pushed, the coverage dropped to 0.

I looked at some threads about coverage but none seemed to be applicable to us. Is there anything we can do to debug this?

Here’s our coverage graph

Thanks

[Colin]

Sounds eerily familar to this post:

Analysis running on old, closed PRs

We are using SonarQube Cloud with GitHub. About 8 hours ago it ran and submitted analysis on a bunch of PRs that were closed more than 2 years ago. We don’t have automatic analysis configured, instead running the analysis in a GitHub action and submitting the results. However, no GitHub action ran around the time the report was submitted. How can I debug what triggered this report to be created?

I’m flagging this for attention and will reach out to you for some additional details (to try and figure out why the analyses were run)

[Colin]

Hey there.

Originally I thought this might be related to another issue of analysis triggering unexpectedly. I no longer think that, so you can ignore my previous post.

A couple of things to check based on your sonar-scanner command and the issues you’re describing with coverage:

1. Let SonarQube/GitLab Handle Branch Names Automatically

You’re currently setting -Dsonar.branch.name=$CI_COMMIT_REF_NAME manually. In the context of GitLab CI, SonarQube is designed to automatically detect branch and PR information through environment variables and the integration with the GitLab pipeline. Setting this property manually can sometimes lead to unexpected results, especially with how analyses are categorized (branches vs PRs vs main).

I recommended removing the -Dsonar.branch.name=$CI_COMMIT_REF_NAME line from your scanner command and let the GitLab integration do the work. This often resolves issues like all analyses getting submitted as branch analysis instead of PR, main, etc.

2. Pipeline Job Configuration

What you’re describing (coverage flipping between 0% and correct values) is often seen when:

• Not all jobs (like tests) are being triggered for all pipelines (e.g., some PRs or branch builds may be skipping the test and coverage export step).
• Coverage reports (coverage.xml) are not always being generated or picked up by the scanner.

Check your CI job definitions to make sure all relevant stages are being run for every relevant pipeline and that the coverage.xml file is present and up to date at the time SonarQube analysis runs. Feel free to share you full Gitlab CI YML here.!

3. Debugging Coverage Import

To dig deeper, check the SonarQube scanner logs for how it’s handling the coverage.xml file in each build. The scanner logs verbose information about what coverage files are found, imported, or missing. This can help you spot differences between runs that succeed vs those that show 0% coverage.

You can find guidance here on how find those logs:

How to find logs about importing code coverage

When coverage data isn’t appearing as expected in SonarQube Server / Community Build or SonarQube Cloud, the first place you should look is the scanner logs. Where do I find the scanner logs? This will likely be broken out into its own guide soon The Scanner logs are the output of executing one of the scanners, either directly or wrapped by one of many integrations, such as the SonarScanner for Jenkins, SonarScanner for Azure DevOps, SonarQube/SonarCloud GitHub Actions or SonarQube /…

Let us know what you find after trying these steps!