Sonar-scanner on front-end project fails to find any unit tests or coverage

SonarQube Server / Community Build
1

We recently upgraded SonarQube from 6.x to 7.9.1. There were a lot of issues to deal with, and I’m still left with at least one more.

Our front-end builds were using a gulp plugin to run the scan, and that was working fine. I don’t remember which plugin we were using, but most recently we were using the one that embeds a copy of sonar-scanner. This was failing due to some weird issue about “not finding zip in path”. I never got an answer to this from any of my queries. In any case, this is moot because I’ve moved on from this.

We are now using a direct call to sonar-scanner in our Jenkinsfile.

This is our resulting sonar-scanner command line, with some elisions:

sonar-scanner -Dsonar.typescript.node=/opt/app/bin/node \
-Dsonar.nodejs.executable=/opt/app/bin/node \
-Dsonar.host.url=http://... \
-Dsonar.login=... \
-Dsonar.password= \
-Dsonar.javascript.lcov.reportPaths=coverage/lcov.info \
-Dsonar.typescript.lcov.reportPaths=coverage/lcov.info \
-Dsonar.branch.name= \
-Dsonar.language=js \
-Dsonar.projectKey=... \
-Dsonar.projectName=... \
'-Dsonar.exclusions=**/*.scss.d.ts, **/*.scss, **/*Props.ts, **/*State.ts, **/*index.ts' \
'-Dsonar.coverage.exclusions=**/*.spec.tsx, **/*.spec.ts, **/*.scss.d.ts, **/*.css.d.ts, **/*.scss' \
-Dsonar.projectVersion=1.0.0 \
'-Dsonar.sources=src/components/,src/api/,src/models/mappers/, src/utils/' \
'-Dsonar.lang.patterns.js=*/.ts,*/.tsx' \
-Dsonar.js.file.suffixes=.ts,.tsx \
-Dsonar.sourceEncoding=UTF-8 \
-Dsonar.tests=src \
'-Dsonar.test.inclusions=**/*.spec.tsx' \
-Dsonar.typescript.tslintconfigpath=tslint.json \
-Dsonar.log.level=DEBUG \
-Dsonar.verbose=true \
'-Dsonar.exec.maxBuffer=1024 * 1024'

The problem we have now is that this is not finding any unit tests and thus not finding any coverage. The resulting sonarqube project does find many lines to cover, but the same number of uncovered lines. In the build process, I run an “ls -lt” of the “coverage” directory, to make sure that the “lcov.info” and associated files are generated. I pass the relative path to that file in the two associated properties in the command line, as seen above.

In the build output, I do see lines like this:

DEBUG: 'src/api/Api.spec.tsx' indexed as test with language 'ts'

So something detected that was a test, but the resulting project has no unit tests. The generated coverage files have coverage for “src/api/Api.tsx”, but this shows no coverage in the generated project.

What could be wrong here?

2

Hi,

I’ve edited your command line to add line breaks. I found it impossible to read as one long line, partly because you’ve got a lot going on there. And I can’t help commenting beyond the scope of your question

I’d swap this to use an analysis token. Safer (and shorter!) in the long run.

This parameter is ignored now, which is a good thing since you’re explicitly trying to analyze TS as well as JS

I don’t recognize that first parameter. With the second, you’re specifying that TS files should be analyzed as JS. Not sure this is what you want, but if it is then your TS-specific coverage reports don’t apply to anything, which is probably part of the problem. Particularly because you’re seeing

Speaking of which, it’s not clear what you expect to happen here. You want both languages to (re)process the same report?

You’ve got a lot going on here and I suspect it’s contributing to your problem. IIWY, I’d whittle this down to just sonar.sources and sonar.tests and get that working, and then build back up - if necessary - from there. And I would do that building back up in the UI.

 
HTH,
Ann

3

There is a bit of cruft there, but the reason I kept it all in for now is that this was all working before the upgrade. One of the risks of property-based configuration is that you can’t tell when settings for obsolete properties become obsolete.

I will try starting with just sources and tests and move on from there.

Concerning the two properties pointing to the lcov.info file, it’s unclear to me what the semantics are for these properties. I thought the point of them was for the processing for those languages to write their coverage data into that file. You appear to imply that the processing direction is somehow reversed from that. If I have both javascript and typescript, what do I need to do here?

Note that we also filed a vendor ticket with you (probably 2 minutes before I noticed your post), SUPPORT-17135 . I might post more voluminous or sensitive output there instead of here.

4

Hi,

Analysis only writes data (ultimately) to the SonarQube server. Almost every analysis parameter is going to apply to data that is read. For coverage, you’ll generate your reports before analysis and then feed the locations of the reports into analysis using those parameters.

Yeah, I get that. I almost withheld comment for that very reason, but couldn’t help myself.

BTW, a lot changes from LTS to LTS, and that can be particularly true with individual languages, as was the case this time with JS and TS. That’s probably what you’re running into here.

 
Ann

5

Ok, then which of those two “lcov” properties do I need to set, as opposed to both of them?

6

Which language do you have test reports for?

7

I’m not sure exactly how to interpret that question, but a different but relevant question might be “which languages do I have unit tests for?”, and the answer is both javascript and typescript.

8

Okay, so your job executes the tests and produces how many test reports?

9

I’m still not sure how to interpret “test reports”. We have both javascript and typescript in the project. We don’t produce separate reports for javascript and typescript. We want to see all the test results, whether javascript or typescript, and we want to see the overall coverage. All of the test and coverage data is integrated into the sonarqube project. We don’t use intermediate reports.

10

As discussed in the docs

SonarSource analyzers do not run your tests or generate reports. They only import pre-generated reports. Below you’ll find language- and tool-specific analysis parameters for importing coverage and execution reports.

In the Guides category of the SonarSource Community forum you might find instructions on generating these reports.

 
HTH,
Ann

11

Ok, so when you refer to a “test report”, you mean the “lcov.info” file. We only generate the one, combining unit test results from both javascript and typescript tests.

12

Ok, now I’ve rerun the scan with the following (some company-specific properties and the token removed):

sonar-scanner -Dsonar.typescript.node=/opt/app/bin/node -Dsonar.nodejs.executable=/opt/app/bin/node \
-Dsonar.token=... -Dsonar.branch.name= -Dsonar.projectKey=... -Dsonar.projectName=... \
-Dsonar.sources=src/components/,src/api/,src/models/mappers/,src/utils/ -Dsonar.tests=src \
'-Dsonar.test.inclusions=**/*.spec.tsx' -Dsonar.log.level=DEBUG -Dsonar.verbose=true \
'-Dsonar.exec.maxBuffer=1024 * 1024'

Unsurprisingly, the resulting SonarQube project has 0% coverage, but over 18k lines to cover. It has no unit tests.

I also note there are many lines like this:

DEBUG: 'src/api/Api.spec.tsx' indexed as test with language 'ts'

I note that I don’t find any lines like this with language “js”, and I realize now that all of the “spec.js” files are in folders that are NOT in the list of folders in the “sonar.sources” list, so that means we’re intending to exclude all of the javascript tests.

I also note that after the block of “indexed as test” lines, there are a bunch of lines that appear to refer to SonarQube plugins. Some of them say “skipped because there is no related file in current project”, which is understandable. Some of them say instead “skipped because one of the required properties is missing”, including “SonarTS Coverage”. That seems pretty significant to me, but it doesn’t say WHICH required properties are missing. I figure at least “sonar.typescript.lcov.reportPaths” would be required, so I’m adding that back (but not the javascript one) and rerunning the build.

13

Well, I have a clue, but I still don’t understand why it’s failing. I finally noticed a line like the following in the build:

WARN: Could not resolve 243 file paths in [/opt/app/.../coverage/lcov.info], \
first unresolved path: /home/.../src/api/Api.tsx

Note that there is a symbolic link from /home/… to /opt/app/… , so that unresolved path does exist, but for some reason SonarTS is not properly evaluating a symbolic link.

When I run “pwd” just before the sonar-scanner call, it says we’re in the “/opt/app/…” path.

I note that the tests run within a container that provides “yarn”, but the sonar-scanner call is run outside of that container. I have no idea whether that’s relevant.

14

I would imagine that this is relevant: "Sensor SonarTS Coverage" fails to match path with symbolic link .

I haven’t yet tried to use the workaround advised there. What I did do, however, is implement a hack post-processing step that does a sed replacement on the lcov.info file, changing all the “/home/…” paths to “/opt/app/…” paths. This gets me closer. I now have 76.5% coverage, which is a little below what we had before the upgrade, but this hasn’t been working for a month or so, so I imagine some developer discipline slipped a bit. However, although it shows coverage, it still doesn’t show any unit tests. In the sonar-scanner command line, I have these:

-Dsonar.tests=src '-Dsonar.test.inclusions=**/*.spec.tsx'

And in the build output, I still see lots of lines like:

'src/api/Api.spec.tsx' indexed as test with language 'ts'

So why aren’t the unit tests shown in the resulting project?

15

I believe I have a solution for the unit tests issue. I’m setting up to use the “jest-sonar-reporter” and the “sonar.testExecutionReportPaths” property. I would really like to understand the problem with the paths in the lcov.info file. I don’t like the hack I had to implement.

18

Hi,

I think that bug report you cited probably identifies the problem. It has been overlooked to date probably because it wasn’t tagged with the language. I’ve fixed that, so hopefully it’ll get picked up soon.

The only remaining question in my mind is whether we should be supporting sym links or we need to document that we don’t.

 
Ann

19

For the record, except for the concern I have about the symlink paths, I now have everything working. I’ve isolated the paths conversion stuff into a method that I can remove if this is ever fixed, although that will likely be with SonarQube 8.1, where lots of other things will change anyway (like moving from SonarTS to SonarJS).