Sonar Kotlin Coverage fails due to "Line is out of range" errors

kotlin
jacoco

(Daniel Kaiser) #1

I use the current version of Sonar with JaCoCo plugin version 1.0.0 and Sonar Kotlin plugin 1.1.
The Sonar scanner I use is version 3.5.0.1254.

When I run Sonar (locally or via CI), I get erorrs like the following:

Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.5.0.1254:sonar (default-cli) on project vcs: Line 79 is out of range in the file src/main/kotlin/example/config/CamelNamingStrategy.kt (lines: 78) -> [Help 1]

This error is based on Kotlin inlining information when it compiles classes:
The Kotlin compiler uses line numbers beyond the file length to denote inlined functions. It has to do that as Java debug information only allows specifying a single source file per class. For Sonar, this has to be “reverted” or at least made not crash on out of bounds line numbers.

It seems that the error is related to the following task in the Sonar Jira board: https://jira.sonarsource.com/browse/SONARSLANG-19 - in my opinion the CoreMetrics.EXECUTABLE_LINES_DATA should be raised for Kotlin files [according to the lines found in the .class file].


Jacoco + Kotlin doesn't seem to work
(Alexandre Gigleux) #3

Hello Daniel,

Do you have a way to share a reproducer through a GitHub repo for example, that will help us a lot to reproduce on our side?
With this reproducer, please provide the steps you are executing to reach this error: build, coverage, code scan commands.

Thanks


(Daniel Kaiser) #4

Hello Alexandre,

I made a very simple program, which shows the issue:


After the commit d3ecc19655e0834b5f2699ac3156301e3fff9c65 Sonar and JaCoCo is working well, after adding an inline function it fails. Of course an explicit inline function is not the only reason why the compiler may add extra lines.
I added the command that I call in the run_sonar.sh shell script; it just builds, tests (with coverage) and runs the scanner.

Thank you


(Alexandre Gigleux) #5

Hello Daniel,

At this stage, we believe a fix should be done on JaCoCo side. Here is the related ticket created by @Godin: https://github.com/jacoco/jacoco/issues/763 with detailed explanations.

Waiting for this fix to come, we are going to patch SonarJaCoCo and release a 1.0.1 version to avoid this failure. The related ticket is: https://jira.sonarsource.com/browse/JACOCO-3.

Regards


Lines out of range in kotlin code
Kotlin Objects not getting coverage
(Robert Stoll) #6

I see that JACOCO-3 is already fixed. In case you have already rolled out 1.0.1 on sonarcloud, I just hit the same error.


(Alexandre Gigleux) #7

Hello,

SonarJaCoCo 1.0.1 is not yet deployed on SonarCloud but it will be soon there.

You can download and install it on your own SonarQube instance from https://docs.sonarqube.org/display/PLUG/Kotlin+Coverage+Results+Import

Regards


(Daniel Kaiser) #8

Thank you, for our project it works and so we have coverage numbers. I will analyze them and wait for the JaCoCo fixed.


(Paul Wellner Bou) #10

Is it supposed to work with JaCoCo 1.0.1?
I have set up a clean SonarQube 7.4 with JaCoCo 1.0.1 and still get this error.

Do I have to do anything special in addition?


(Alexandre Gigleux) #11

Hello,

With SonarJaCoCo 1.0.1 your analysis should not fail, log a message to explain why this data can’t be loaded and continue. Is your analysis stopped when you have the error?

I suggest to double-check that you really have sonar-jacoco-plugin-1.0.1.143.jar in your $SQ_HOME/extensions/plugins directory. If this is the case, I believe the best to continue would be to share your logs, your project if possible or a reproducer so we can debug that.

Thanks


(Paul Wellner Bou) #12

Hello,

thank you. No, the analysis does not fail, but the stacktraces are still there.
Makes sense, I assume this is how it should behave (until the JaCoCo issue is fixed)?

The measures displayed are correct then? (more or less… disregarding the lines mentioned in the bug ticket of JaCoCo)

Thanks and Regards!


(Paul Wellner Bou) #13

The numbers are very different. We have a (very small) project where JaCoCo locally measures 71% and sonarqube tells us 39%. Not sure if this is due to this JaCoCo bug or any other issue.

@Alexandre_Gigleux: Should we expect a difference like this?


(Colin Mueller) #14

As these measures are available at the file level in SonarQube, you should be able to compare your local reports with those in SonarQube.

Note that SonarQube defines on its own what can be tested (which lines are executable), so for example, if you have a module with no coverage and no coverage report, that will be taken into consideration when SonarQube calculates total code coverage.


(David Marin) #15

it does look like jacoco works differently than the sonar jacoco plugin. I also have different results in the jacoco html report and in the sonar one. The reason seems to be that in the jacoco-sonar plugin when there is an exception the coverage for the whole file is 0, but the jacoco html still maps the valid lines.

The detekt plugin seems to do the same (just ignore the lines that can not be matched against the .kt source file)

Seems that the original problem is given by the kotlin compiler, but for what is being discussed here looks like it wont be fixed --> https://youtrack.jetbrains.com/issue/KT-9766

A PR ignoring the invalid lines can be found here: https://github.com/SonarSource/sonar-jacoco/pull/10


(Marcel van den Brink) #16

I am experiencing the same behaviour as @David_Marin is mentioning on Sonarcloud. On my project it generates valid reports in HTML, but as soon as they are uploaded the coverage is set to zero on the files that show the stack traces in the logs.

This is showing wrong values on the dashboard on Sonarcloud… and even fails the quality gate. Navigating to the code of shows that nothing is being tested, while I am 100% sure that it is. :slight_smile:


(Evgeny Mandrikov) #18

@David_Marin @Leviter please use JaCoCo version 0.8.3 that contains fix for https://github.com/jacoco/jacoco/issues/763


(Marcel van den Brink) #19

Yes! That works perfectly.
Thanks for the follow-up.