Strange code coverage result switching from jacoco to scoverage

  • CI system used: Concourse
  • Scanner command used: mvn sonar:sonar
  • Languages of the repository: Scala

I have a strange codecoverage result in my sonarcloud project after switching codecoverage from jacoco to scoverage.

I inherited a project which was always built in scala, however codecoverage was being collected by jacoco. This is problematic because it does not deal with generated scala code very well (for example, case classes), therefore we decided to switch to a scala tool, scoverage.

I deleted the jacoco settings in sonarcloud, we no longer send jacoco reports and we now send scoverage reports. I also deleted old analyses.

Despite taking the steps above I see a codecoverage number which bears no meaning when compared with the scoverage report, and is suspiciously similar to the value from the last jacoco report.

Is there some step I may have missed when trying to change the codecoverage report generator?

Is there some way I can see where coverage is being drawn from (the analyses in the activity page did not help. Do I need to delete them ALL? I only deleted those with an actual code coverage metric)

project is SonarCloud

properties in pom.xml:

<scoverage.plugin.version>1.4.11</scoverage.plugin.version>
<sonar.language>scala</sonar.language>
<sonar.sources>src/main/scala</sonar.sources>
<sonar.scala.coverage.reportPaths>
target/scoverage.xml
</sonar.scala.coverage.reportPaths>

Hey there.

What do the logs say about which reports are being imported?

Hi, I actually found the post you linked to yesterday but it wasn’t that helpful for maven, the subsequently linked page, ie here SonarScanner for Maven doesn’t say how to get logs for the maven scanner, unless I am overlooking something

Anyway, I suspect that somehow the jacoco file is still being sent and I am currently investigating that.

Is there a way to exclude it from being considered via the sonarcloud project settings?
Is there a way to see what was used to build the coverage report in sonarcloud?

Never, I figured it out, for anyone reading I added <sonar.log.level>DEBUG</sonar.log.level> to the properties in my pom.xml

So, I ran the maven sonar scanner locally, passing ONLY my scoverage.xml file. the jacoco report does not exist locally

From the logs it seems like sonar insists on also looking for a jacoco file (I included only what seemed relevant log lines):

[INFO] — sonar:3.9.1.2184:sonar (default-cli) @ cube-activity-triggers-import —
[INFO] SonarQube version: 8.0.0.43832
[INFO] Default locale: “en_GB”, source code encoding: “UTF-8”
[INFO] Load global settings
[INFO] Load global settings (done) | time=312ms
[INFO] Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
[INFO] User cache: /Users/kat/.sonar/cache
[INFO] Load/download plugins
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=318ms
[INFO] Load/download plugins (done) | time=435ms
[INFO] Loaded core extensions: developer-scanner
[INFO] Load project settings for component key: ‘tech.stage.cube:cube-activity-triggers-import’
[INFO] Load project settings for component key: ‘tech.stage.cube:cube-activity-triggers-import’ (done) | time=244ms
[INFO] Process project properties
[INFO] Execute project builders
[INFO] Execute project builders (done) | time=1ms
[INFO] Project key: tech.stage.cube:cube-activity-triggers-import
[INFO] Base dir: /Users/kat/work/ice/code/cube-activity-triggers-import
[INFO] Working dir: /Users/kat/work/ice/code/cube-activity-triggers-import/target/sonar
[INFO] Load project branches
[INFO] Load project branches (done) | time=280ms
[INFO] Check ALM binding of project ‘tech.stage.cube:cube-activity-triggers-import’
[INFO] Detected project binding: NOT_BOUND
[INFO] Check ALM binding of project ‘tech.stage.cube:cube-activity-triggers-import’ (done) | time=237ms
[INFO] Load project pull requests
[INFO] Load project pull requests (done) | time=368ms
[INFO] Load branch configuration
[INFO] Load branch configuration (done) | time=1ms
[INFO] Load quality profiles
[INFO] Load quality profiles (done) | time=331ms
[INFO] Load active rules
[INFO] Load active rules (done) | time=5600ms
[INFO] Organization key: icerepository
[INFO] Branch name: sonar-jacoco-missing-test, type: short-lived
[INFO] Load project repositories
[INFO] Load project repositories (done) | time=259ms
[INFO] SCM collecting changed files in the branch
[INFO] SCM collecting changed files in the branch (done) | time=144ms
[INFO] Indexing files…
[INFO] Project configuration:
[INFO] Excluded sources: **/build-wrapper-dump.json
[INFO] 49 files indexed
[INFO] 0 files ignored because of inclusion/exclusion patterns
[INFO] 0 files ignored because of scm ignore settings
[INFO] Quality profile for scala: Sonar way
[INFO] ------------- Run sensors on module cube-activity-triggers-import
[INFO] Load metrics repository
[INFO] Load metrics repository (done) | time=261ms
[INFO] Sensor cache enabled
[INFO] Load sensor cache
[INFO] Load sensor cache (404) | time=232ms
[INFO] Sensor Scala Sensor [sonarscala]
[INFO] 31 source files to be analyzed
[INFO] 31/31 source files have been analyzed
[INFO] Sensor Scala Sensor [sonarscala] (done) | time=1943ms
[INFO] Sensor Scoverage sensor for Scala coverage [sonarscala]
[INFO] Importing /Users/kat/work/ice/code/cube-activity-triggers-import/target/scoverage.xml
[INFO] Sensor Scoverage sensor for Scala coverage [sonarscala] (done) | time=115ms
…snipped…
[INFO] Sensor JaCoCo XML Report Importer [jacoco]
[INFO] ‘sonar.coverage.jacoco.xmlReportPaths’ is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
[INFO] No report imported, no coverage information will be imported by JaCoCo XML Report Importer
[INFO] Sensor JaCoCo XML Report Importer [jacoco] (done) | time=0ms
…snipped…
[INFO] Analysis report generated in 94ms, dir size=258 KB
[INFO] Analysis report compressed in 38ms, zip size=78 KB
[INFO] Analysis report uploaded in 449ms
[INFO] ANALYSIS SUCCESSFUL, you can find the results at: SonarCloud
[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[INFO] More about the report processing at https://sonarcloud.io/api/ce/task?id=AYlO2xtJ_SFmdtvEltiU
[INFO] Analysis total time: 13.087 s

Why is it looking for jacoco? I tried through the sonarcloud project settings to make it clear it is a scala project. Will sonar always look for all sorts of coverage files?

Based on the logs it’s finding an scoverage report, but not a jacoco repoort.

So no JaCoCo report should be getting in the way.

Can you find a specific discrepancy between the scoverage report and the report on SonarQube? This might help narrow down where the issue is coming from.

Hi, sorry I was not clear. I tried this locally to understand what is going on and was surprised to see it looking for jacoco.

In our builds, which I cannot modify, both scoverage and jacoco reports are available. Therefore I was asking if there is anyway in the project configuration on sonarcloud that I can get it to ignore jacoco

It is ignoring JaCoCo.

[**INFO** ] No report imported, no coverage information will be imported by JaCoCo XML Report Importer

this log is from is the local run, not the build.

In my local run I have no jacoco file.

In the build, which I cannot change, there is a file and sonarcloud appears to load it/treat it as the main coverage file.

This is despite me trying to change the settings in sonarcloud to force it to only care about scoverage.

So I was asking if, if there is a jacoco file present, it always take precedence or if there is a way to tell sonarcloud to ignore it.

Thank you for clarifying.

At this point, if you don’t want the JaCoCo reports because they don’t deal with generated code well… why not stop generating the reports altogether? This way they won’t exist at the default location, and SonarCloud won’t pick them up.

(A perfectly valid reason is “Because I want / have to” – and I understand it’s not a smooth experience having JaCoCo be included when you don’t want it to, but I do have to ask first)

As I wrote before jacoco is generated in our builds and I unfortunately cannot change them, so I cannot stop it from being generated

Hi @kat-sharp-ice,

The problem seems related to the path where the reports were generated.
Can you please check if your sonar.scala.coverage.reportPaths property is set correctly?

All the best,

Irina

1 Like

I have
<sonar.scala.coverage.reportPaths>
target/scoverage.xml
</sonar.scala.coverage.reportPaths>

in my pom.xml and also in the UI I have (see screenshot)

And can you please confirm that scoverage reports are generated on this path?

Scoverage reports are usually generated in target/scala-X.x/scoverage-report/scoverage.xml.
Can you also try setting the path to a similar format?

1 Like