GitHub Actions code coverage reverts locally produced coverage in Swift project

SonarQube Cloud
, , ,
1

Template for a good new topic, formatted with Markdown:

  • ALM used: GitHub
  • CI system used: GitHub Actions
  • Scanner command used when applicable:
Run ${GITHUB_ACTION_PATH}/scripts/run-sonar-scanner-cli.sh 
  ${GITHUB_ACTION_PATH}/scripts/run-sonar-scanner-cli.sh 
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    SONAR_TOKEN: ***
    INPUT_PROJECTBASEDIR: 
    SONAR_SCANNER_JRE: /home/runner/work/_temp/sonar-scanner-cli-7.1.0.4889-Linux-X64/jre
  • Languages of the repository: Swift
  • Only if the SonarCloud project is public, the URL: Example project
  • Error observed (wrap logs/code around with triple quotes ``` for proper formatting)
09:22:48.311 INFO  Scanner configuration file: /home/runner/work/_temp/sonar-scanner-cli-7.1.0.4889-Linux-X64/conf/sonar-scanner.properties
09:22:48.315 INFO  Project root configuration file: /home/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/sonar-project.properties
09:22:48.330 INFO  SonarScanner CLI 7.1.0.4889
09:22:48.332 INFO  Java 17.0.13 Eclipse Adoptium (64-bit)
09:22:48.332 INFO  Linux 6.11.0-1012-azure amd64
09:22:48.363 INFO  User cache: /home/runner/.sonar/cache
09:22:49.004 INFO  Communicating with SonarQube Cloud
09:22:49.005 INFO  JRE provisioning: os[linux], arch[x86_64]
09:22:52.463 INFO  Starting SonarScanner Engine...
09:22:52.464 INFO  Java 17.0.11 Eclipse Adoptium (64-bit)
09:22:53.261 INFO  Load global settings
09:22:53.982 INFO  Load global settings (done) | time=724ms
09:22:53.986 INFO  Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
09:22:54.170 INFO  Loading required plugins
09:22:54.170 INFO  Load plugins index
09:22:54.365 INFO  Load plugins index (done) | time=195ms
09:22:54.365 INFO  Load/download plugins
09:22:55.047 INFO  Load/download plugins (done) | time=682ms
09:22:55.387 INFO  Found an active CI vendor: 'Github Actions'
09:22:55.393 INFO  Load project settings for component key: 'jbehrens94_dojo.smelly-mars-rover'
09:22:55.873 INFO  Load project settings for component key: 'jbehrens94_dojo.smelly-mars-rover' (done) | time=481ms
09:22:55.878 INFO  Process project properties
09:22:55.885 INFO  Project key: jbehrens94_dojo.smelly-mars-rover
09:22:55.885 INFO  Base dir: /home/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover
09:22:55.885 INFO  Working dir: /home/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/.scannerwork
09:22:55.890 INFO  Load project branches
09:22:56.157 INFO  Load project branches (done) | time=267ms
09:22:56.159 INFO  Check ALM binding of project 'jbehrens94_dojo.smelly-mars-rover'
09:22:56.276 INFO  Detected project binding: BOUND
09:22:56.276 INFO  Check ALM binding of project 'jbehrens94_dojo.smelly-mars-rover' (done) | time=116ms
09:22:56.277 INFO  Load project pull requests
09:22:56.442 INFO  Load project pull requests (done) | time=165ms
09:22:56.444 INFO  Load branch configuration
09:22:56.445 INFO  Github event: push
09:22:56.447 INFO  Auto-configuring branch main
09:22:56.448 INFO  Load branch configuration (done) | time=4ms
09:22:56.455 INFO  Load quality profiles
09:22:56.866 INFO  Load quality profiles (done) | time=411ms
09:22:56.871 INFO  Load active rules
09:23:06.249 INFO  Load active rules (done) | time=9378ms
09:23:06.447 INFO  Organization key: jbehrens94
09:23:06.448 INFO  Branch name: main, type: long-lived
09:23:06.461 INFO  Preprocessing files...
09:23:06.534 INFO  1 language detected in 3 preprocessed files
09:23:06.534 INFO  0 files ignored because of inclusion/exclusion patterns
09:23:06.534 INFO  0 files ignored because of scm ignore settings
09:23:06.590 INFO  Loading plugins for detected languages
09:23:06.591 INFO  Load/download plugins
09:23:06.623 INFO  Load/download plugins (done) | time=33ms
09:23:06.660 INFO  Load project repositories
09:23:06.889 INFO  Load project repositories (done) | time=230ms
09:23:06.892 INFO  Indexing files...
09:23:06.892 INFO  Project configuration:
09:23:06.893 INFO    Excluded sources: **/build-wrapper-dump.json
09:23:06.899 INFO  3 files indexed
09:23:06.902 INFO  Quality profile for swift: Sonar way
09:23:06.902 INFO  ------------- Run sensors on module MarsRover
09:23:06.922 INFO  Load metrics repository
09:23:07.069 INFO  Load metrics repository (done) | time=147ms
09:23:07.073 INFO  Sensor cache enabled
09:23:07.206 INFO  Load sensor cache
09:23:07.983 INFO  Load sensor cache (404) | time=777ms
09:23:08.272 INFO  Sensor JaCoCo XML Report Importer [jacoco]
09:23:08.272 INFO  'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
09:23:08.272 INFO  No report imported, no coverage information will be imported by JaCoCo XML Report Importer
09:23:08.273 INFO  Sensor JaCoCo XML Report Importer [jacoco] (done) | time=2ms
09:23:08.273 INFO  Sensor Java Config Sensor [iac]
09:23:08.295 INFO  0 source files to be analyzed
09:23:08.306 INFO  0/0 source files have been analyzed
09:23:08.307 INFO  Sensor Java Config Sensor [iac] (done) | time=34ms
09:23:08.307 INFO  Sensor Swift Code Quality and Security [swift]
09:23:08.312 INFO  2 source files to be analyzed
09:23:09.102 INFO  2/2 source files have been analyzed
09:23:09.103 INFO  Sensor Swift Code Quality and Security [swift] (done) | time=796ms
09:23:09.103 INFO  Sensor IaC Docker Sensor [iac]
09:23:09.168 INFO  0 source files to be analyzed
09:23:09.169 INFO  0/0 source files have been analyzed
09:23:09.170 INFO  Sensor IaC Docker Sensor [iac] (done) | time=64ms
09:23:09.171 INFO  Sensor Generic Coverage Report
09:23:09.172 INFO  Parsing /home/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/coverage.xml
09:23:09.199 INFO  Imported coverage data for 0 files
09:23:09.201 INFO  Coverage data ignored for 2 unknown files, including:
/Users/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/Sources/MarsRoverKit/MarsRoverKit.swift
/Users/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/Tests/MarsRoverTests/MarsRoverTests.swift
09:23:09.201 INFO  Sensor Generic Coverage Report (done) | time=31ms
09:23:09.201 INFO  Sensor EnterpriseTextAndSecretsSensor [textenterprise]
09:23:09.202 INFO  Available processors: 4
09:23:09.202 INFO  Using 4 threads for analysis.
09:23:09.667 INFO  Using git CLI to retrieve untracked files
09:23:09.770 INFO  Analyzing language associated files and files included via "sonar.text.inclusions" that are tracked by git
09:23:09.779 INFO  3 source files to be analyzed
09:23:09.830 INFO  3/3 source files have been analyzed
09:23:09.831 INFO  Sensor EnterpriseTextAndSecretsSensor [textenterprise] (done) | time=630ms
09:23:09.832 INFO  ------------- Run sensors on project
09:23:09.937 INFO  Sensor Zero Coverage Sensor
09:23:09.941 INFO  Sensor Zero Coverage Sensor (done) | time=5ms
09:23:09.948 INFO  CPD Executor 1 file had no CPD blocks
09:23:09.948 INFO  CPD Executor Calculating CPD for 1 file
09:23:09.952 INFO  CPD Executor CPD calculation finished (done) | time=5ms
09:23:13.296 INFO  Analysis report generated in 3341ms, dir size=288 KB
09:23:13.316 INFO  Analysis report compressed in 20ms, zip size=60 KB
09:23:13.728 INFO  Analysis report uploaded in 412ms
09:23:13.729 INFO  ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=jbehrens94_dojo.smelly-mars-rover&branch=main
09:23:13.729 INFO  Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
09:23:13.729 INFO  More about the report processing at https://sonarcloud.io/api/ce/task?id=AZaGAeY72bPTfLnOPvl7
09:23:13.732 INFO  Analysis total time: 18.519 s
09:23:13.733 INFO  SonarScanner Engine completed successfully
09:23:14.067 INFO  EXECUTION SUCCESS
09:23:14.068 INFO  Total time: 25.759s

Question

If I have a local clone of the repository I mentioned above, and I run sonar-scanner locally after generating coverage.xml, it will push 82.9% of coverage to Sonarcloud. However, if I run it from my pipeline on GitHub Actions, it will revert back to 0.0%.

The error that strikes me is:

09:23:09.172 INFO  Parsing /home/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/coverage.xml
09:23:09.199 INFO  Imported coverage data for 0 files
09:23:09.201 INFO  Coverage data ignored for 2 unknown files, including:
/Users/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/Sources/MarsRoverKit/MarsRoverKit.swift
/Users/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/Tests/MarsRoverTests/MarsRoverTests.swift

Which means that the coverage.xml file doesn’t seem to match what Sonarcloud wants. However, locally, I get this log:

11:24:50.523 INFO  Parsing /Users/jbehrens/Developer/Dojo.smelly-mars-rover/coverage.xml
11:24:50.548 INFO  Imported coverage data for 2 files

What could be going on?

2

Hey there.

Can you share your full GitHub Actions YML file?

3

Hey Colin, thank you for your quick response! I’ll add two files, one where the workflow is fine. The other where it’s broken.

Broken workflow

on:
  push:
    branches: [main]
  schedule:
    - cron: 0 0 * * *

jobs:
  lint:
    runs-on: macos-15
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: SwiftLint
        uses: cirruslabs/swiftlint-action@v1
        with:
          version: latest
  test:
    runs-on: macos-15
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Test
        run: xcodebuild test -scheme MarsRover -enableCodeCoverage YES -resultBundlePath ./build.xcresult -destination "platform=macOS,arch=arm64e,variant=macos" -skipPackagePluginValidation
      - name: Convert to Sonarcube generic
        run: ./scripts/xccov-to-sonarqube-generic.sh ./build.xcresult > coverage.xml
      - name: SonarQube
        uses: SonarSource/sonarqube-scan-action@v5.1.0
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Working flow

on:
  push:
    branches: [main]
  schedule:
    - cron: 0 0 * * *

jobs:
  lint:
    runs-on: macos-15
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: SwiftLint
        uses: cirruslabs/swiftlint-action@v1
        with:
          version: latest
  test:
    runs-on: macos-15
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Install sonar-scanner
        run: brew install sonar-scanner
      - name: Test
        run: xcodebuild test -scheme MarsRover -enableCodeCoverage YES -resultBundlePath ./build.xcresult -destination "platform=macOS,arch=arm64e,variant=macos" -skipPackagePluginValidation
      - name: Convert to Sonarcube generic
        run: ./scripts/xccov-to-sonarqube-generic.sh ./build.xcresult > coverage.xml
      - name: Push to SonarQube
        run: sonar-scanner
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
4

Thanks for that. I’m actually pretty surprised – I was expecting to see an older version of the sonarqube-scan-action (one that only runs in a linux environment).

What stands out to me from your original logs is that within the runner, you seem to be working in /home/runner/

09:23:09.172 INFO  Parsing /home/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/coverage.xml

But your coverage report is referencing /Users/runner/

09:23:09.201 INFO  Coverage data ignored for 2 unknown files, including:
/Users/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/Sources/MarsRoverKit/MarsRoverKit.swift
/Users/runner/work/dojo.smelly-mars-rover/dojo.smelly-mars-rover/Tests/MarsRoverTests/MarsRoverTests.swift

Can you check the actual coverage.xml to see if it’s using an absolute path with /Users/, and if it’s different in your working pipeline?

5

Thank you for your hint. This has already helped a lot. I downloaded a version of the coverage.xml file from an older actions run and it indeed uses the macOS path /Users/runner/....

So I imagine if I produce the coverage.xml on macOS, upload the artifact or pass it on to the action that runs on ubuntu-latest instead of macOS, it stops working. Should the sonarqube-scan-action work on macOS? When I tried it before, it didn’t detect my SONAR_TOKEN through the GitHub runner secret.

6

Checking back in after trying out something. I thought: let’s use the sonarqube-scan-action on macos-latest, because then the paths from coverage.xml will correctly map. However, I hope you might be able to see the logs from this run. It’s just one simple set-up ci.yml file, but it doesn’t seem to get the SONAR_TOKEN.

Run ${GITHUB_ACTION_PATH}/scripts/run-sonar-scanner-cli.sh 
  ${GITHUB_ACTION_PATH}/scripts/run-sonar-scanner-cli.sh 
  shell: /bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    SONAR_TOKEN: ***
    INPUT_PROJECTBASEDIR: 
    SONAR_SCANNER_JRE: /Users/runner/work/_temp/sonar-scanner-cli-7.1.0.4889-macOS-ARM64/jre
/Users/runner/work/_actions/SonarSource/sonarqube-scan-action/v5.1.0/scripts/run-sonar-scanner-cli.sh: line 80: scanner_args[@]: unbound variable

Were that to be fixed - assuming it’s an issue with the sonarqube action - I think everything would work. However, the tricky part here is: to get the coverage, I assume I have to use xcodebuild. (For the xccov-to-generic-SonarQube format script) That will generate macOS paths in the coverage.xml file, which then can only be uploaded through sonar-scanner on macOS. Unless there’s a different path that I don’t know of, but I hope you can enlighten me about that. :slight_smile:

7

Hey @jbehrens94

I assumed from everything you posted before that you were indeed generating the coverage report new on each run. That wasn’t the case?

Based on the logs shared, it seems you’re running into this known issue SQSCANGHA-83. We’ve known about the issue since February so I’m a bit sad to see it’s not fixed yet. :frowning:

That said I believe you can get around this by adding

SONARCLOUD_URL: sonarcloud.io

to the env section of your workflow (right after SONAR_TOKEN)

8

I assumed from everything you posted before that you were indeed generating the coverage report new on each run. That wasn’t the case?

I am indeed generating the coverage.xml report every new run. However, I assume the issue was in running tests on macOS + generating the coverage.xml, then uploading the artifact and using it in an ubuntu-latest runner. Then the path wouldn’t match anymore.

That said I believe you can get around this by adding

SONARCLOUD_URL: sonarcloud.io

to the env section of your workflow (right after SONAR_TOKEN)

If I add your suggestion for SONARCLOUD_URL, I get new errors:

Run ${GITHUB_ACTION_PATH}/scripts/run-sonar-scanner-cli.sh 
  ${GITHUB_ACTION_PATH}/scripts/run-sonar-scanner-cli.sh 
  shell: /bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    SONAR_TOKEN: ***
    SONARCLOUD_URL: sonarcloud.io
    INPUT_PROJECTBASEDIR: 
    SONAR_SCANNER_JRE: /Users/runner/work/_temp/sonar-scanner-cli-7.1.0.4889-macOS-ARM64/jre
+ sonar-scanner -Dsonar.scanner.sonarcloudUrl=sonarcloud.io
18:32:32.356 INFO  Scanner configuration file: /Users/runner/work/_temp/sonar-scanner-cli-7.1.0.4889-macOS-ARM64/conf/sonar-scanner.properties
18:32:32.358 INFO  Project root configuration file: /Users/runner/work/dojo.mars-rover/dojo.mars-rover/sonar-project.properties
18:32:32.366 INFO  SonarScanner CLI 7.1.0.4889
18:32:32.366 INFO  Java 17.0.13 Eclipse Adoptium (64-bit)
18:32:32.367 INFO  Mac OS X 15.4.1 aarch64
18:32:32.370 INFO  EXECUTION FAILURE
18:32:32.370 INFO  Total time: 0.016s
18:32:32.370 ERROR Error during SonarScanner CLI execution
org.sonarsource.scanner.lib.internal.MessageException: Defining a custom 'sonar.scanner.sonarcloudUrl' without providing 'sonar.scanner.apiBaseUrl' is not supported.
	at org.sonarsource.scanner.lib.internal.endpoint.ScannerEndpointResolver.resolveCustomSonarQubeCloudEndpoint(ScannerEndpointResolver.java:73)
	at org.sonarsource.scanner.lib.internal.endpoint.ScannerEndpointResolver.resolveSonarQubeCloudEndpoint(ScannerEndpointResolver.java:47)
	at org.sonarsource.scanner.lib.internal.endpoint.ScannerEndpointResolver.resolveEndpoint(ScannerEndpointResolver.java:41)
	at org.sonarsource.scanner.lib.ScannerEngineBootstrapper.bootstrap(ScannerEngineBootstrapper.java:131)
	at org.sonarsource.scanner.cli.Main.analyze(Main.java:76)
	at org.sonarsource.scanner.cli.Main.main(Main.java:64)
18:32:32.371 ERROR 
18:32:32.371 ERROR Re-run SonarScanner CLI using the -X switch to enable full debug logging.
Error: Process completed with exit code 1.