Struggling to get Sonarqube to parse jest lcov.info content for coverage in a javascript/spring boot project

SonarQube
1

SonarQube server version CE 7.9.6

Problem: Cannot get sonarqube to report test coverage using jest on a javascript/node/springboot codebase.

Goal: Get SonarQube to report the contents of the lcov.info file on the sonar project panel

Actions Summary: Using the maven-frontend-plugin I have successfully implemented and integrated the jest test

In my project root folder is the client folder where the lcov.info file is being written
[projectroot]/deconweb/coverage/lcov.info

Scanner context for the most recent scan (Note the context for the lcov file is the full path in this run not the relative path which i have also tried)

SonarQube plugins:
  - SonarPython 1.14.1.3143 (python)
  - SonarCSS 1.1.1.1010 (cssfamily)
  - GitHub Authentication for SonarQube 1.5.0.870 (authgithub)
  - JaCoCo 1.0.2.475 (jacoco)
  - SonarGo 1.1.1.2000 (go)
  - SonarKotlin 1.5.0.315 (kotlin)
  - Svn 1.9.0.1295 (scmsvn)
  - SonarJS 5.2.1.7778 (javascript)
  - SonarRuby 1.5.0.315 (ruby)
  - SonarScala 1.5.0.315 (sonarscala)
  - SonarC# 7.15.0.8572 (csharp)
  - SonarJava 5.13.1.18282 (java)
  - LDAP 2.2.0.608 (ldap)
  - SonarHTML 3.1.0.1615 (web)
  - Git 1.8.0.1574 (scmgit)
  - SonarFlex 2.5.1.1831 (flex)
  - SonarXML 2.0.1.2020 (xml)
  - SAML 2.0 Authentication for SonarQube 1.2.0.682 (authsaml)
  - SonarPHP 3.2.0.4868 (php)
  - SonarTS 1.9.0.3766 (typescript)
  - SonarVB 7.15.0.8572 (vbnet)
Global server settings:
  - sonar.core.id=F6267474-AXPuU55WReNP70gyi4b1
  - sonar.core.serverBaseURL=http://tressel.tdkc.com:9000
  - sonar.core.startTime=2021-03-08T20:00:17+0000
  - sonar.leak.period=5
Project server settings:
  - sonar.javascript.lcov.reportPaths=/home/jenkins/workspace/decon-web_buildtest/deconweb/coverage/lcov.info
Project scanner properties:
  - sonar.binaries=/home/jenkins/workspace/decon-web_develop/target/classes
  - sonar.groovy.binaries=/home/jenkins/workspace/decon-web_develop/target/classes
  - sonar.host.url=http://tressel.tdkc.com:9000
  - sonar.java.binaries=/home/jenkins/workspace/decon-web_develop/target/classes
  - sonar.java.jdkHome=/usr/local/openjdk-8
  - sonar.java.libraries=/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter-data-mongodb/2.2.6.RELEASE/spring-boot-starter-data-mongodb-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter/2.2.6.RELEASE/spring-boot-starter-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot/2.2.6.RELEASE/spring-boot-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.2.6.RELEASE/spring-boot-autoconfigure-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.2.6.RELEASE/spring-boot-starter-logging-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar,/home/jenkins/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar,/home/jenkins/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.12.1/log4j-to-slf4j-2.12.1.jar,/home/jenkins/.m2/repository/org/apache/logging/l...
  - sonar.java.source=1.8
  - sonar.java.target=1.8
  - sonar.java.test.libraries=/home/jenkins/workspace/decon-web_develop/target/classes,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter-data-mongodb/2.2.6.RELEASE/spring-boot-starter-data-mongodb-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter/2.2.6.RELEASE/spring-boot-starter-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot/2.2.6.RELEASE/spring-boot-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.2.6.RELEASE/spring-boot-autoconfigure-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.2.6.RELEASE/spring-boot-starter-logging-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar,/home/jenkins/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar,/home/jenkins/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.12.1/log4j-to-slf4j-2.1...
  - sonar.libraries=/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter-data-mongodb/2.2.6.RELEASE/spring-boot-starter-data-mongodb-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter/2.2.6.RELEASE/spring-boot-starter-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot/2.2.6.RELEASE/spring-boot-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.2.6.RELEASE/spring-boot-autoconfigure-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.2.6.RELEASE/spring-boot-starter-logging-2.2.6.RELEASE.jar,/home/jenkins/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar,/home/jenkins/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar,/home/jenkins/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.12.1/log4j-to-slf4j-2.12.1.jar,/home/jenkins/.m2/repository/org/apache/logging/l...
  - sonar.links.ci=
  - sonar.links.homepage=https://projects.spring.io/spring-boot/#/spring-boot-starter-parent/decon-web
  - sonar.links.issue=
  - sonar.links.scm=https://github.com/spring-projects/spring-boot/spring-boot-starter-parent/decon-web
  - sonar.links.scm_dev=
  - sonar.login=******
  - sonar.moduleKey=decon-web:com.tdkc.decon:decon-web
  - sonar.projectBaseDir=/home/jenkins/workspace/decon-web_develop
  - sonar.projectBuildDir=/home/jenkins/workspace/decon-web_develop/target
  - sonar.projectDescription=DECON Web App
  - sonar.projectKey=decon-web
  - sonar.projectName=decon-web
  - sonar.projectVersion=0.0.1-SNAPSHOT
  - sonar.scanner.app=ScannerMaven
  - sonar.scanner.appVersion=3.8.0.2131/3.3.9
  - sonar.sourceEncoding=UTF-8
  - sonar.sources=/home/jenkins/workspace/decon-web_develop/pom.xml,/home/jenkins/workspace/decon-web_develop/src/main/java
  - sonar.tests=/home/jenkins/workspace/decon-web_develop/src/test/java
  - sonar.working.directory=/home/jenkins/workspace/decon-web_develop/target/sonar

The output from the sonar scan stage of my jenkins build is as follows

+ mvn sonar:sonar -Dsonar.projectKey=decon-web -Dsonar.host.url=http://********.*******.com:9000 -Dsonar.login=4882501d2a46705c3194a0c8d9686262fbec80d2
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building decon-web 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- sonar-maven-plugin:3.8.0.2131:sonar (default-cli) @ decon-web ---
[INFO] User cache: /home/jenkins/.sonar/cache
[INFO] SonarQube version: 7.9.6
[INFO] Default locale: "en_US", source code encoding: "UTF-8"
[WARNING] SonarScanner will require Java 11 to run starting in SonarQube 8.x
[INFO] Load global settings
[INFO] Load global settings (done) | time=122ms
[INFO] Server id: F6267474-AXPuU55WReNP70gyi4b1
[INFO] User cache: /home/jenkins/.sonar/cache
[INFO] Load/download plugins
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=73ms
[INFO] Load/download plugins (done) | time=106ms
[INFO] Process project properties
[INFO] Execute project builders
[INFO] Execute project builders (done) | time=4ms
[INFO] Project key: decon-web
[INFO] Base dir: /home/jenkins/workspace/decon-web_buildtest
[INFO] Working dir: /home/jenkins/workspace/decon-web_buildtest/target/sonar
[INFO] Load project settings for component key: 'decon-web'
[INFO] Load project settings for component key: 'decon-web' (done) | time=19ms
[INFO] Load quality profiles
[INFO] Load quality profiles (done) | time=72ms
[INFO] Detected Jenkins
[INFO] Load active rules
[INFO] Load active rules (done) | time=710ms
[INFO] Indexing files...
[INFO] Project configuration:
[INFO] 4 files indexed
[INFO] 0 files ignored because of scm ignore settings
[INFO] Quality profile for java: Sonar way
[INFO] Quality profile for xml: Sonar way
[INFO] ------------- Run sensors on module decon-web
[INFO] Load metrics repository
[INFO] Load metrics repository (done) | time=23ms
[INFO] Sensor JavaSquidSensor [java]
[INFO] Configured Java source version (sonar.java.source): 8
[INFO] JavaClasspath initialization
[INFO] JavaClasspath initialization (done) | time=12ms
[INFO] JavaTestClasspath initialization
[INFO] JavaTestClasspath initialization (done) | time=4ms
[INFO] Java Main Files AST scan
[INFO] 2 source files to be analyzed
[INFO] Load project repositories
[INFO] Load project repositories (done) | time=29ms
[INFO] 2/2 source files have been analyzed
[INFO] Java Main Files AST scan (done) | time=515ms
[INFO] Java Test Files AST scan
[INFO] 1 source files to be analyzed
[INFO] 1/1 source files have been analyzed
[INFO] Java Test Files AST scan (done) | time=50ms
[INFO] Sensor JavaSquidSensor [java] (done) | time=1525ms
[INFO] Sensor JaCoCo XML Report Importer [jacoco]
[INFO] Sensor JaCoCo XML Report Importer [jacoco] (done) | time=2ms
[INFO] Sensor SurefireSensor [java]
[INFO] parsing [/home/jenkins/workspace/decon-web_buildtest/target/surefire-reports]
[INFO] Sensor SurefireSensor [java] (done) | time=2ms
[INFO] Sensor JaCoCoSensor [java]
[INFO] Sensor JaCoCoSensor [java] (done) | time=1ms
[INFO] Sensor JavaXmlSensor [java]
[INFO] 1 source files to be analyzed
[INFO] Sensor JavaXmlSensor [java] (done) | time=149ms
[INFO] Sensor HTML [web]
[INFO] 1/1 source files have been analyzed
[INFO] Sensor HTML [web] (done) | time=12ms
[INFO] Sensor XML Sensor [xml]
[INFO] 1 source files to be analyzed
[INFO] Sensor XML Sensor [xml] (done) | time=130ms
[INFO] 1/1 source files have been analyzed
[INFO] ------------- Run sensors on project
[INFO] Sensor Zero Coverage Sensor
[INFO] Sensor Zero Coverage Sensor (done) | time=8ms
[INFO] Sensor Java CPD Block Indexer
[INFO] Sensor Java CPD Block Indexer (done) | time=12ms
[INFO] 2 files had no CPD blocks
[INFO] Calculating CPD for 0 files
[INFO] CPD calculation finished
[INFO] Analysis report generated in 63ms, dir size=96 KB
[INFO] Analysis report compressed in 19ms, zip size=20 KB
[INFO] Analysis report uploaded in 24ms
[INFO] ANALYSIS SUCCESSFUL, you can browse http://******.******.com:9000/dashboard?id=decon-web
[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[INFO] More about the report processing at http://******.******.com:9000/api/ce/task?id=AXhkVeVfkmwf1BNHdfJM
[INFO] Analysis total time: 4.899 s
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.132 s
[INFO] Finished at: 2021-03-24T13:04:14+00:00
[INFO] Final Memory: 86M/3205M
[INFO] ------------------------------------------------------------------------

Jenkins flow
Pull code
Build using maven build with maven-frontend-plugin to build and run the jest tests
Then run the maven sonar:sonar goal as mentioned at the top of the log above.
Console shows 0% coverage and the above log mentions nothing about parsing a lcov.info file

What am I missing here?

2

Hi,

Your properties list doesnā€™t seem to include sonar.javascript.lcov.reportPaths ā€¦?

Ā 
Ann

3

Hi Ann
Thank you for the reply. I saw that as well but was confused because I have a path configured in the UI configuration for the javascript lcov files. Is that just not getting picked up or is there something additional I need to add manually somewhere?

image
image799Ɨ120 4.03 KB

4

I had been playing with this value trying to get it to work using both absolute and relative pathing. But maybe it was for naught if it was not being picked up by the job itself.

5

Hi,

Why donā€™t you add -Dsonar.scanner.dumpToFile=[path to file] so we can see what properties analysis is actually getting?

Ā 
Ann

6

Hi Ann
Thank you again for taking the time to look at it this with me
Please find attached the dump file.
The dump does look like it it is not picking up the config change for the project to reference the lcov files dump.txt (72.8 KB)

7

Hi,

Can you try adding it to the analysis properties explicitly? Also, this dump file includes the analysis token you just revokedā€¦ right?

Ā 
Ann

8

Lol 100% right after I pulled the dump.
Can you illuminate for me how I would add the lcov to the analysis properties explicitly? Still fairly new to sonar

9

Is it as simple as adding -Dsonar.javascript.lcov.reportPaths=path to the command line?

10

Hi,

Yes. You can add a property to your pom or add it on the command line like you did the dump parameter.

Ā 
Ann

11

Good morning Ann
Thank you again for your help.
I was able to get the value passed into the scanner but it does not appear to be parsing the file.
in the scanner context on the server for this project I now see
sonar.javascript.lcov.reportPaths=deconweb/coverage/lcov.info
The relative path to the lcov.info file, relative to the jenkins workspace is
deconweb/coverage/lcov.info
The full path to the file is /home/jenkins/workspace/decon-web_buildtest/deconweb/coverage/lcov.info
please find attached the execution output from the build
what should I see in the output if it would be picking up and parsing the file?
latestrun.txt (4.7 KB)

12

Hi,

Iā€™m not seeing anything about JavaScript analysis in your log. Letā€™s back up. Youā€™re trying to get JavaScript test coverage results imported in your analysis, but have you succeeded in getting the JavaScript analyzed?

Ā 
Ann

13

That is probably a wise thing to do. To this point no, I have seen no javascript analysis in our process.
We have a web app written using nodeJS/vue.js and its wrapped in a spring cloud based service.
We use Jest to functionally test the node based web application.
We are using the frontend-maven-plugin to facilitate the node build and test execution within the normal maven build process. The build being managed by jenkins running in a container executing the build on a separate build agent container.
once the build is complete we fire an additional stage to handle the sonar scan.

Prior to working on the node stuff I have been successfully analyzing and capturing coverage for our java based services.

Very happy to provide you any additional information regarding this as I am at a loss and hours of searching has led to very little progress

14

I may be revealing some ignorance in using sonarqube but from what I had found in my searches. Everything indicated that all that was needed was to point sonar to the coverage file and it would consume and report. But based on your comment above it seems like maybe I need to have sonar actually analyze both the java AND the javascript content before it can apply the coverage analysis?

15

So I dug a little deeper based on your comment.
I discovered that my sonar.sources value points only to the pom in the root and the java source location

sonar.sources=/home/jenkins/workspace/decon-web_buildtest/pom.xml,/home/jenkins/workspace/decon-web_buildtest/src/main/java

As such I dont think it would ever even see the javascript files which are located in the following directory

/home/jenkins/workspace/decon-web_buildtest/deconweb

which is where the package.json file is located. Though I cannot figure out how to include that path for the sources value

16

Hi,

Not surprising, actually. Youā€™re doing a Maven analysis, which is fine since your Java build is Maven, but that means that itā€™s Java-centric to the point of ignoring other languages. You need to override the default sonar.sources value Maven provides either on the command line or in a pom property.

Generally true, butā€¦

Yes. Why? Well, ā€œcoverageā€ of a text file, or an image or any other non-source file is really meaningless. So until/unless thereā€™s source to go along with it, thereā€™s no need to read/process the coverage report.

 
HTH,
Ann

17

Good Morning Ann
Thank you for the reply. Th lightbulb is starting to turn on here, thanks to you.
I am making some progress now.
Adding the path to the js code to the sonar.sources property in the pom I now see that the scanner can see the js code but it skips them all with the following message for each of the js file

/home/jenkins/workspace/decon-web_develop/deconweb/node_modules/core-js/proposals/array-unique.jsā€™ is ignored. It is not located in project basedir ā€˜/home/jenkins/workspace/decon-web_buildtestā€™.

My project structure looks like this
[decon-web_buildtest] (workspace root)
|ā€”src (java source)
| |ā€”main etc.
|ā€”deconweb
| |ā€”src (js source)

Also if I may ask another question about order or precedence. In what order do the property overrides process? I am making an assumption that is the following could you confirm or correct?

server-> project->sonarscan.properties->pom->commandline

where each item above overrides the previous so commandline would override all?

Also I am a little confused with he sonarscan.properties file. Where does that get created/maintained? Is it in the same location as the pom file? Do I have to indicate to the scanner to look for it it, or does it get picked up if its present.

18

Hi,

Iā€™m confused.

The project structure you outline doesnā€™t seem to match what you passed in as the source directory.

Parameter precedence is documented at the top of the Analysis Parameters page.

For Maven analysis you donā€™t need a sonar-project.properties file. Its purpose is to provide to analysis the properties that are being read automatically from your project in a Maven context.

 
Ann

19

Sincerest apologies
I grabbed the wrong branch path, I have changed the pathing to relative pathing and BOOM I have analysis and coverage in console.
Thank you so much for your guidance, Once I wrap my head around everything I will post a consolidated response with what I did with your guidance to get this working for future folks who may be struggling

1 Like
20

Hi Ann
I do have one more follow on if I may
We have test execution reports being generated by the jest-sonar-reporter plugin - its writing the file test-report.xml and I added the necessary property in my pom file
<sonar.testExecutionReportPaths>deconweb/test-report.xml</sonar.testExecutionReportPaths>
It is picking up the file but ignoring the results with the following message?
Something stupid I am missing here?

[INFO] Parsing /home/jenkins/workspace/decon-web_FinalBuildTest/deconweb/test-report.xml
[INFO] Imported test execution data for 0 files
[INFO] Test execution data ignored for 33 unknown files, including:
/home/jenkins/workspace/decon-web_FinalBuildTest/deconweb/ tests /RulesUtil.test.js
/home/jenkins/workspace/decon-web_FinalBuildTest/deconweb/ tests /DateUtil.test.js
/home/jenkins/workspace/decon-web_FinalBuildTest/deconweb/ tests /Logger.test.js
/home/jenkins/workspace/decon-web_FinalBuildTest/deconweb/ tests /TableFunctions.test.js
/home/jenkins/workspace/decon-web_FinalBuildTest/deconweb/ tests /Helpers.test.js
[INFO] Sensor Generic Test Executions Report (done) | time=8ms