We have two layers of testing over the same source files that produce two different lcov report files.
What I’m experiencing is that sonarcloud is not merging the coverages. I want to make sure this is expected behavior.
Just to ensure I’m clear let me illustrate with an oversimplified example:
example.js has two functions.
The first one is tested with our first layer of tests giving a coverage of 50%
The second one is tested with the second layer of tests with a coverage of 50%
My expectation would be to see 100% coverage in sonar but I see 50% (and only seems to be using the first lcov)
To add a bit more info. We are importing more than 2 lcov files (we have a mono repo) and It does pick up all of them as I have coverage for all my source files. This is the only case where two different layers of tests run over the same source files.
I’ve tried to merge the lcov files before sending them to sonar but the tools out there seem to be outdated and not doing the work well.
Just to make sure, you’re passing both coverage reports into analysis at the same time, right? You’re not passing in one and then re-analyzing with the second one, are you?
Same time yes. And again, it’s picking up all the report files, except in this specific case where two reports describe the coverage of the same files.
Yes, it’s a reasonable expectation. Double-checking the docs:
Wildcards and a comma-delimited list of paths are supported.
So, multiple reports are expected. (This can also be seen in the name of the parameter: sonar.javascript.lcov.reportPaths. Altho it would be reasonable to discount this, since some other parameter names are ransom-seeming.)
Well, but again. That’s not the same thing. Admitting multiple reports is not the same as sonar being able to merge reports that affect same files. I have not found anything in the documentation of sonar that says that it’s able of doing that.
We’ve had several reports working properly for a long time already. So that’s not where my question was headed anyways.
Can SonarCloud combine multiple test coverage files for the same source files in a way that increases the overall coverage beyond the coverage of each individual file?
We are in a similar situation, we are migrating away from one test framework to another, so in the mean time we have lcov files generated into two different folders. As the old framework has a few unit tests, when I have added a test using the old framework, I see the lcov it generates has some coverage. Whereas the lcov files based on the new framework have far less coverage (this is normal).
We use a glob in the sonar properties to properly target the lcov files, and I believe I see them loaded from the logs I see from the sonar scanner.
Here is what we see in our lcov report summary from the generated html:
When it comes to the sonarcloud report, we see 0% coverage on this particular file, where it should be at least greater than zero:
If we enable the verbose flag in sonar.properties, I notice that there are a handful of files that seem to have this error after loading of the lcov files:
Problem during processing LCOV report: can't save DA data for line 36536 of coverage report file (java.lang.IllegalArgumentException: Line with number 404 doesn't belong to file
I tried to navigate to line 36536 but the suspected lcov files do not have this amount of lines, so I’m guessing they are merged / concatenated at some point in the process.
Is there a way to dump the lcov file that the sonar scan references?
@lop , thanks a lot for sharing the logs and the LCOV reports.
In the logs, there are a lot of error messages about a file named hubspot.service.ts:
(java.lang.IllegalArgumentException: Line with number 549 doesn't belong to file hubspot.service.ts).
There is one error per line, from line 82 to line 549. This matches what the LCOV reports report about the file src/hubspot/hubspot.service.ts: it consists of 549 lines.
I did a test with a file named src/hubspot/hubspot.service.ts consisting of only 82 lines, which I analyzed using the two LCOV reports that you provided (sonar.javascript.lcov.reportPaths=lcov_1.info,lcov_2.info), and I get the exact same errors in my logs.
I suspect that the file src/hubspot/hubspot.service.ts consists only of 82 lines.
Can you please check this assumption and let me know?
@lop , sorry for the delay. We have been trying to reproduce your issue without success. Could you please put a public repository in place with a minimal reproducible example so that we can investigate and provide support?
