TFS 2018 SonarQube CLI task

patdelaney · December 9, 2019, 6:48pm

Must-share information (formatted with Markdown):

which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension) 6.75
what are you trying to achieve : Command line scanner in TFS
what have you tried so far to achieve this

I am trying to configure the Perform a SonarQube analysis (SonarQube Scanner CLI) using the extension from the TFS Market Place for TFS 2018.
What I have setup in TFS is Build that is just going to use the CLI scanner to scan source (the REPO) whenever something is committed. The project in TFS Git is primarily JSON Config files, but there is are also Javascript files, with the .sjs extension.

What I seem to be having an issue with is defining what files to pull into the scanner via the CLI task. I’m not sure what should be defined in Sources. I put $(Build.Repository.LocalPath). But I’m not sure that is correct.

When I run the job it completes, but when I look at SonarQube Web UI I don’t think its scanning the files.
Do we have to define the extension of .sjs to be javascript?

When I run the scanner it only says 2 files scanned and they were both .xml files. In the whole project there are more than 2 xml files.

Here is the Log

******************************************************************************
Starting: Perform a SonarQube analysis
******************************************************************************
==============================================================================
Task         : SonarQube  Scanner CLI
Description  : Perform a SonarQube analysis of the source code of your PHP, JavaScript... projects
Version      : 3.1.0
Author       : sonarsource
Help         : Version: 3.1.0. [More Information](http://redirect.sonarsource.com/doc/install-configure-scanner-tfs-ts.html)
==============================================================================
Preparing task execution handler.
Executing the powershell script: E:\Dev\AgtProd-service\_work\_tasks\SonarQubeScannerCli_9f57024b-31f9-4e58-9e39-a47ccc098f03\3.1.0\SonarQubeScanner.ps1
INFO: Scanner configuration file: E:\Dev\AgtProd-service\_work\_tasks\SonarQubeScannerCli_9f57024b-31f9-4e58-9e39-a47ccc098f03\3.1.0\sonar-scanner\bin\..\conf\sonar-scanner.properties
INFO: Project root configuration file: NONE
INFO: SonarQube Scanner 3.0.3.778
INFO: Java 1.8.0_231 Oracle Corporation (64-bit)
INFO: Windows 10 10.0 amd64
INFO: User cache: C:\Users\TFSBuild1_SVCACCT\.sonar\cache
INFO: Publish mode
INFO: Load global settings
INFO: Load global settings (done) | time=109ms
INFO: Server id: 68653A6C-128cb948359873e
INFO: User cache: C:\Users\TFSBuild1_SVCACCT\.sonar\cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=31ms
INFO: SonarQube server 6.7.5
INFO: Default locale: "en_US", source code encoding: "windows-1252" (analysis is platform dependent)
INFO: Process project properties
INFO: Execute project builders
INFO: Execute project builders (done) | time=0ms
INFO: Load branch configuration
INFO: Load branch configuration (done) | time=0ms
INFO: Load project repositories
INFO: Load project repositories (done) | time=62ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=31ms
INFO: Load active rules
INFO: Load active rules (done) | time=844ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=16ms
INFO: Project key: DataHub
INFO: -------------  Scan DataHub
INFO: Load server rules
INFO: Load server rules (done) | time=110ms
INFO: Base dir: E:\Dev\AgtProd-service\_work\4\s
INFO: Working dir: E:\Dev\AgtProd-service\_work\4\s\.scannerwork
INFO: Source paths: .
INFO: Source encoding: windows-1252, default locale: en_US
INFO: Index files
INFO: 182 files indexed
INFO: Sensor JavaXmlSensor [java]
INFO: 2 source files to be analyzed
INFO: Sensor JavaXmlSensor [java] (done) | time=125ms
INFO: Sensor HTML [web]
INFO: 2/2 source files have been analyzed
INFO: Sensor HTML [web] (done) | time=16ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=15ms
INFO: Sensor CPD Block Indexer
INFO: Sensor CPD Block Indexer (done) | time=0ms
INFO: Calculating CPD for 0 files
INFO: CPD calculation finished
INFO: Analysis report generated in 78ms, dir size=62 KB
INFO: Analysis reports compressed in 63ms, zip size=17 KB
INFO: Analysis report uploaded in 31ms
INFO: ANALYSIS SUCCESSFUL, you can browse https://sonar-corp-p1.com/dashboard/index/DataHub
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at https://sonar-corp-p1.com/api/ce/task?id=AW7r7p-e-zTPeRMvMhN8
INFO: Task total time: 3.469 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 4.844s
INFO: Final Memory: 67M/435M
INFO: ------------------------------------------------------------------------
******************************************************************************
Finishing: Perform a SonarQube analysis
******************************************************************************

mickaelcaro · December 10, 2019, 7:54am

Hi @patdelaney,

In the global settings of your SonarQube instance, you can configure which extension will be taken into account for javascript

You can add there your .sjs and try.

By the way, you are using pretty old version of both the Scanner and SonarQube itself, we highly recommend you to updgrade to at least the latest LTS (7.9) as yours are no longer supported.

Thank you !

Mickaël

patdelaney · December 10, 2019, 12:46pm

I will follow up with our Admins and find out why we are on such an old version. I did go to my Test instance of TFS 2018 and removed the SonarQube extension. The extension was originally installed when we were using TFS 2015. We did the
in-place upgrade to TFS 2018.

In the dev instance, I deleted the extension and then reinstalled it. It looks like it pulled in a newer version of the build steps. I see 3 of them.

Prepare analysis of SonarQube
Run Code Analysis
Publish Quality Gate Result

I no longer see the Perform a SonarQube analysis (SonarQube Scanner CLI) is that by default? So for my particular workflow where I just want to scan the java script code that is checked in would I just create a new build that downloads
the sources and then add the 3 steps?

mickaelcaro · December 10, 2019, 12:53pm

Hi,

You can have a look at our documentation here : https://docs.sonarqube.org/latest/analysis/scan/sonarscanner-for-azure-devops/

Basically, the prepare configuration step let you configure how the task will call the Scanner itself : In standalone mode (that is the one you used to use), with MSBuild, and so on…

You can there setup the properties that you want to pass to the scanner, as you may have done that way in the previous version.

Mickaël

patdelaney · December 10, 2019, 1:22pm

The admin added .sjs as a valid extension. I reran the scan job, the job passed but there was this error in the console window. What does it mean?

mickaelcaro · December 10, 2019, 1:52pm

I don’t see the error, just a logo, can you check ?

Thanks !

patdelaney · December 10, 2019, 2:37pm

Sorry about that, here you go. The error I’m asking about is: ERROR: Error when running: ‘node -v’. Is Node.js available during analysis?

Do I need to install node.js on the windows server that is running the build?

mickaelcaro · December 10, 2019, 2:46pm

Yes. You will need all the tool that the CI tasks need to build your project (java, node if applicable, …)

patdelaney · December 10, 2019, 5:27pm

So I am trying the new scanner v4 in my test instance of TFS 2018 and test instance of SonarQube.

I created a SonarQube token and defined that in the TFS Services for SonarQube.

When I run the job I get the following error.

mickaelcaro · December 11, 2019, 6:50am

Hi,

I still don’t see the error message that you get. Can you check please ?

Thank you.

patdelaney · December 11, 2019, 12:39pm

I am now using the V4 plugin and it’s having issues with getting the version.

Starting: Prepare analysis on SonarQube

==============================================================================
Task : Prepare Analysis Configuration
Description : Prepare SonarQube analysis configuration
Version : 4.8.0
Author : sonarsource
Help : More Information

agent.TempDirectory=E:\Dev\Agent-svc_work_temp
loading inputs and endpoints
loading ENDPOINT_AUTH_8f36bfb6-b06d-43a0-951d-34af7848628d
loading ENDPOINT_AUTH_PARAMETER_8f36bfb6-b06d-43a0-951d-34af7848628d_USERNAME
loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
loading ENDPOINT_AUTH_SCHEME_8f36bfb6-b06d-43a0-951d-34af7848628d
loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
loading INPUT_CLIPROJECTVERSION
loading INPUT_CLISOURCES
loading INPUT_CONFIGFILE
loading INPUT_CONFIGMODE
loading INPUT_EXTRAPROPERTIES
loading INPUT_PROJECTKEY
loading INPUT_PROJECTNAME
loading INPUT_PROJECTVERSION
loading INPUT_SCANNERMODE
loading INPUT_SONARQUBE
loaded 16
Agent.ProxyUrl=undefined
Agent.CAInfo=undefined
Agent.ClientCert=undefined
Agent.SkipCertValidation=undefined
SonarQube=8f36bfb6-b06d-43a0-951d-34af7848628d
8f36bfb6-b06d-43a0-951d-34af7848628d=https://sonar-corp-d1.wpsic.com/
8f36bfb6-b06d-43a0-951d-34af7848628d auth param apitoken = null
8f36bfb6-b06d-43a0-951d-34af7848628d auth param username = ***
8f36bfb6-b06d-43a0-951d-34af7848628d auth param password = null
organization=null
scannerMode=CLI
configMode=file
configFile=E:\Dev\Agent-svc_work\7\s\sonar-project.properties
[SQ] API GET: ‘/api/server/version’ with query “undefined”
[SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}
task result: Failed
[SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}
Processed: ##vso[task.issue type=error;][SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}
Processed: ##vso[task.complete result=Failed;][SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}

Finishing: Prepare analysis on SonarQube

Evaluating condition for step: ‘Post Job Cleanup’
Evaluating: AlwaysNode()
Evaluating AlwaysNode:
=> True
Result: True

Starting: Post Job Cleanup

mickaelcaro · December 11, 2019, 1:30pm

Which version of SonarQube did you manage to install ?

Sounds like your SQ instance is exposed with an SSL certificate, which might no be installed (at least the root) in your build machine.

patdelaney · December 11, 2019, 2:00pm

API version is : 6.7.5.38563
I’m using the latest version of the VSTS SonarQube plugin.

The SonarQube Server is on a Linux Server, we do use self-signed Certs. I had already added that Cert to the PC that is running the TFS agent.

We do not have Node.js installed on the PC that is running the TFS agent. I stumbled across a note that said we may need an intermediate cert I added that and rebooted the PC still no joy.

The Older version of the VSTS plugin (v3) worked fine

mickaelcaro · December 11, 2019, 2:03pm

Node.js is used by the build agent itself on its internal library to make REST API calls against external services.

You can try to define the path to the cert by modifying this environment variable (it expects one or multiple paths) : NODE_EXTRA_CA_CERTS

Mickaël

patdelaney · December 11, 2019, 2:06pm

Do I add that to the sonar-project.properties file? This is what I have in the file at this point:

#----- Default SonarQube server
#sonar.host.url=http://localhost:9000

Here is the Job as defined in TFS

mickaelcaro · December 11, 2019, 2:07pm

No, it’s directly in the system environment variables of the machine where your build agent is installed on.

patdelaney · December 11, 2019, 2:14pm

I tried that and it still fails

2019-12-11T14:11:07.1481718Z ##[debug]task result: Failed
2019-12-11T14:11:07.1510169Z ##[error][SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}
2019-12-11T14:11:07.1517636Z ##[debug]Processed: ##vso[task.issue type=error;][SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}
2019-12-11T14:11:07.1525795Z ##[debug]Processed: ##vso[task.complete result=Failed;][SQ] API GET ‘/api/server/version’ failed, error was: {“code”:“UNABLE_TO_VERIFY_LEAF_SIGNATURE”}
2019-12-11T14:11:07.1590001Z ##[section]Finishing: Prepare analysis on SonarQube

patdelaney · December 11, 2019, 2:15pm

do I need to install Node.js on the build server?

mickaelcaro · December 11, 2019, 2:23pm

Normally no, the build agent is taking care of downloading a specific version.

Do you have the entire certificate path inside the .cer ? With root and intermediate ?

patdelaney · December 11, 2019, 3:22pm

API version is : 6.7.5.38563

I’m using the latest version of the VSTS SonarQube plugin.

The SonarQube Server is on a Linux Server, we do use self-signed Certs. I had already added that Cert to the PC that is running the TFS agent.

We do not have Node.js installed on the PC that is running the TFS agent. I stumbled across a note that said we may need an intermediate cert I added that and rebooted the PC still no joy.

The Older version of the VSTS plugin (v3) worked fine

TFS 2018 SonarQube CLI task

============================================================================== Task : Prepare Analysis Configuration Description : Prepare SonarQube analysis configuration Version : 4.8.0 Author : sonarsource Help : More Information

==============================================================================
Task : Prepare Analysis Configuration
Description : Prepare SonarQube analysis configuration
Version : 4.8.0
Author : sonarsource
Help : More Information