Pytest coverage not showing up in sonarcloud

SonarQube Cloud
1
  • ALM used : GitHub
  • CI system used : GitHub Actions
  • Scanner command used when applicable : using sonarqube-scan-action
  • Languages of the repository : python
  • Error observed :
**Coverage**
There is not enough lines to compute coverage

Heres a snippet of our ci build file. We run pytest and generate xml reports using pytest-cov :

  unit-test:
    name: Run Unit Tests
    runs-on: ubuntu-24.04-4core
    timeout-minutes: 60
    strategy:
      fail-fast: false
      matrix:
        test:
          - test-cli
          - test-policies
        python-version: ${{ fromJson(inputs.python-versions) }}

    steps:
      - name: Check out code
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Prepare the job for unit tests
        uses: ./.github/actions/test-prerequisites
        with:
            PYTHON_VERSION: ${{ matrix.python-version }}
            POETRY_CACHE_VERSION: ${{ secrets.POETRY_CACHE_VERSION }}

      - name: Install Dependencies (Linux) 📦
        run: |
          make prepare-tests-ubuntu

      - name: Test Code 🔍 (multi-process)
        env:
          JOBS: 6
          PYTHONUTF8: 1
          API_DOCS_FORMAT: yaml
        run: |
          poetry run pytest tests -n $(JOBS) --dist loadscope -m "$(PYTEST_MARKER)" --cov test --cov-report xml:unit-tests-results.xml
        shell: bash # bash shell is a way to make code run for both Linux and Windows

      - name: Prepare test results
        if: always()
        run: |
          mv unit-tests-results.xml ${{ github.workspace }}/unit-tests-results-${{ matrix.test }}-python-${{ matrix.python-version }}.xml


      - name: Save Test Results
        if: always()
        uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 #v4.4.3
        with:
          name: unit-tests-results-${{ matrix.test }}-python-${{ matrix.python-version }}
          path: |
            ${{ github.workspace }}/unit-tests-results-${{ matrix.test }}-python-${{ matrix.python-version }}.xml

  sonarqube:
    name: SonarQube
    runs-on: ubuntu-24.04
    needs: [ unit-test ]
    if: always()
    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
        with:
          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis
      - name: Download test reports
        uses: actions/download-artifact@c850b930e6ba138125429b7e5c93fc707a7f8427 #v4.1.4
        with:
          path: ${{ github.workspace }}/artifacts
          pattern: "unit-tests-results-**"
          merge-multiple: true

      - name: Check if reports are empty
        run: |
          cat ${{ github.workspace }}/artifacts/unit-tests-results-test-policies-python-3.9.xml

      - name: SonarQube Scan
        uses: SonarSource/sonarqube-scan-action@bfd4e558cda28cda6b5defafb9232d191be8c203 # v4.2.1
        env:
          GITHUB_TOKEN: ${{ secrets.R_GITHUB_TOKEN }} 
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

sonar-project.properties file :

sonar.projectKey=***
sonar.organization=***
sonar.python.version=3.9
sonar.python.coverage.reportPaths=/home/runner/work/<project_name>/artifacts/unit-tests-results-test-policies-python-3.9.xml,\
  /home/runner/work/<project_name>/artifacts/unit-tests-results-test-cli-python-3.9.xml

If you see in the attached logs below, the report xml files are correctly parsed.

15:09:34.033 INFO  Starting rules execution
15:09:34.036 INFO  1274 source files to be analyzed
15:09:44.039 INFO  61/1274 files analyzed, current file: tests/shared/importers/test_multi_project.py
15:09:54.037 INFO  202/1274 files analyzed, current file: rasa/utils/tensorflow/metrics.py
15:10:04.037 INFO  342/1274 files analyzed, current file: rasa/dialogue_understanding/commands/knowledge_answer_command.py
15:10:14.037 INFO  529/1274 files analyzed, current file: tests/dialogue_understanding/stack/frames/__init__.py
15:10:24.039 INFO  699/1274 files analyzed, current file: tests/dialogue_understanding/commands/__init__.py
15:10:34.038 INFO  936/1274 files analyzed, current file: rasa/e2e_test/e2e_test_runner.py
15:10:44.038 INFO  1152/1274 files analyzed, current file: tests/tracing/instrumentation/test_single_step_llm_command_generator_instrumentation.py
15:10:48.753 INFO  1274/1274 source files have been analyzed
15:10:48.754 INFO  The Python analyzer was able to leverage cached data from previous analyses for 0 out of 1274 files. These files were not parsed.
15:10:48.754 INFO  Sensor Python Sensor [python] (done) | time=99253ms
15:10:48.754 INFO  Sensor Cobertura Sensor for Python coverage [python]
15:10:51.720 INFO  Python test coverage
15:10:51.721 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-policies-python-3.9.xml'
15:10:51.762 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-cli-python-3.9.xml'
15:10:51.769 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-nlu-featurizers-python-3.9.xml'
15:10:51.777 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-nlu-predictors-python-3.9.xml'
15:10:51.778 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-full-model-training-python-3.9.xml'
15:10:51.780 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-other-unit-tests-python-3.9.xml'
15:10:51.819 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-performance-python-3.9.xml'
15:10:51.820 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-flaky-python-3.9.xml'
15:10:51.822 INFO  Parsing report '/home/runner/work/rasa-private/rasa-private/artifacts/unit-tests-results-test-anonymization-python-3.9.xml'
15:10:51.822 INFO  Sensor Cobertura Sensor for Python coverage [python] (done) | time=3068ms
15:10:51.822 INFO  Sensor PythonXUnitSensor [python]
15:10:52.179 INFO  Sensor PythonXUnitSensor [python] (done) | time=357ms
15:10:52.179 INFO  Sensor JaCoCo XML Report Importer [jacoco]
15:10:52.182 INFO  'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml

The xml report files are not empty and i have added a step in the CI to cat into the xml files and verify that its not empty.
I am failing to understand why I still don’t see a coverage in sonarcloud and instead see a There is not enough lines to compute coverage message.

2

Hi,

Where are you seeing that message? Is it in the New Code tab or in a PR analysis? Conditions on coverage and duplications on New Code aren’t applied when there are fewer than 20 new lines. (There’s a note on this in the docs.)

 
HTH,
Ann

3

Hi Ann,

Thanks for replying. I see the There is not enough lines to compute coverage message in the PR Summary page. I also have conditions and coverage set for overall coverage, so I expect to see some coverage percentage and not 0. Or what am I doing wrong here? Adding screenshots for you for reference.

Screenshot 2025-03-07 at 17.24.47
Screenshot 2025-03-07 at 17.24.472860Ă—1306 203 KB

Screenshot 2025-03-07 at 17.24.57
Screenshot 2025-03-07 at 17.24.571838Ă—862 45.1 KB

4

Hi,

What do you see on the Code tab of that PR? What were the changes in that PR? Were they all non-code changes such as comments and whitespace, or perhaps configuration files?

 
Ann

5

Hi,

The Code tab is empty. The PR contained only config files since I was adding the sonar config into our repository.

6

Hi,

PR analysis only shows what’s relevant to what changed in the PR. Since there were no code changes in the PR, then it’s quite normal not to show any coverage (since non-code can’t be covered).

 
HTH,
Ann