Using SonarQube with Azure DevOps build pipelines on a self-hosted windows pc

SonarQube Server / Community Build
,
1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • what are you trying to achieve
  • what have you tried so far to achieve this

Hello everyone. I was directed here by a sonarSource Business developer in order to get some help with my case. I am trying to integrate SonarQube static analysis on C/C++ code into my DevOps Build pipeline and I am not succeeding in getting this work.

For clarity, I am currently only looking at a evaluation license of SonarQube. But if I can’t get that to work, then there is no sense in buying it.

Our setup
We are a small development team and we need to write automotive grade software (hence the need for static analysis). We currently develop using visual studio and our code is store on DevOps. We are evolving our development environment towards CI and thus, we are setting up build pipelines to automatically build code and documentation, run unit tests and perform static analysis on pull requests towards the main branches. For this, we have refurbished a windows 10 Pro computer that is running in our network and is connected to the internet. We have setup the pc as a self-hosted worker for DevOps build pipelins using these instructions:

This works. The code builds, the unit tests run, …

What am I trying to do
We want to include SonarQube into this process. Hence, we want the local worker to call the SonarQube analysis after the build. We want to keep this low cost, so we were hoping to avoid having to rent another server or service to run the SonarQube on, but instead we want to run it locally on the pc.

What I did already
In installed SonarQube (version 8.2.0) on the windows PC using these instructions:

For this I also installed java. I am using the internal database of SonarQube for the testing.
Running the StartSonar.bat file gives:

wrapper | → Wrapper Started as Console
wrapper | Launching a JVM…
jvm 1 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
jvm 1 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
jvm 1 |
jvm 1 | 2020.05.09 15:14:48 INFO app[o.s.a.AppFileSystem] Cleaning or creating temp directory C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp
jvm 1 | 2020.05.09 15:14:48 INFO app[o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
jvm 1 | 2020.05.09 15:14:48 INFO app[o.s.a.ProcessLauncherImpl] Launch process[[key=‘es’, ipcIndex=1, logFilenamePrefix=es]] from [C:\Program Files\sonarqube-8.2.0.32929\elasticsearch]: C:\Program Files\Java\jdk-13.0.2\bin\java -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp -XX:ErrorFile=…/logs/es_hs_err_pid%p.log -Xmx512m -Xms512m -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=C:\Program Files\sonarqube-8.2.0.32929\elasticsearch -Des.path.conf=C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp\conf\es -cp lib/* org.elasticsearch.bootstrap.Elasticsearch
jvm 1 | 2020.05.09 15:14:48 INFO app[o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm 1 | Java HotSpot™ 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
jvm 1 | 2020.05.09 15:14:48 INFO app[o.e.p.PluginsService] no modules loaded
jvm 1 | 2020.05.09 15:14:48 INFO app[o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm 1 | 2020.05.09 15:14:58 INFO app[o.s.a.SchedulerImpl] Process[es] is up
jvm 1 | 2020.05.09 15:14:58 INFO app[o.s.a.ProcessLauncherImpl] Launch process[[key=‘web’, ipcIndex=2, logFilenamePrefix=web]] from [C:\Program Files\sonarqube-8.2.0.32929]: C:\Program Files\Java\jdk-13.0.2\bin\java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttp.nonProxyHosts=localhost|127.|[::1] -cp ./lib/common/;C:\Program Files\sonarqube-8.2.0.32929\lib\jdbc\h2\h2-1.4.199.jar org.sonar.server.app.WebServer C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp\sq-process6405025995804454429properties
jvm 1 | 2020.05.09 15:15:13 INFO app[o.s.a.SchedulerImpl] Process[web] is up
jvm 1 | 2020.05.09 15:15:13 INFO app[o.s.a.ProcessLauncherImpl] Launch process[[key=‘ce’, ipcIndex=3, logFilenamePrefix=ce]] from [C:\Program Files\sonarqube-8.2.0.32929]: C:\Program Files\Java\jdk-13.0.2\bin\java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp --add-opens=java.base/java.util=ALL-UNNAMED -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttp.nonProxyHosts=localhost|127.|[::1] -cp ./lib/common/;C:\Program Files\sonarqube-8.2.0.32929\lib\jdbc\h2\h2-1.4.199.jar org.sonar.ce.app.CeServer C:\Program Files\sonarqube-8.2.0.32929\var\sonarqube\temp\sq-process16950362565889781576properties
jvm 1 | 2020.05.09 15:15:18 INFO app[o.s.a.SchedulerImpl] Process[ce] is up
jvm 1 | 2020.05.09 15:15:18 INFO app[o.s.a.SchedulerImpl] SonarQube is up

So, this says that the sonarQube is running and indeed, if I then access localhost:9000 I see the SonarQube website. I communicated the serverID to the business developer and he provided an evaluation license which was accepted by the local SonarQube.
So far so good, except that I have the question if this means that I need to make sure that the ‘startsonar.bat’ file is always running?

DevOps
Next I turned to DevOps and followed the instructions on: SonarQube extension for Azure DevOps
First I need to create a Service-connection. Here, I already run into an issue. What do I need to provide for the Server URL? Does it need to be accessible from DevOps (in which case I might also run into firewall issues)? Or does it need to be accessible by the local worker?

Filling in the pc’s local IP address and ignoring this further, I went on to add the ‘Prepare analysis on SonarQube’ task to my build pipeline. I point to the endpoint.
Then, I don’t really know whether to choose ‘integrate with MSBuild’ or ‘use standalone scanner’?

I tried both, but when I run the pipeline, I immediatly get an error:

[error]No agent found in pool which satisfies the following demand: java

So, at this point, I no longer know how to continue or what I am doing wrong. I get the feeling that I am using SonarQube in a way that it was not ment to be used, but I understood that SonarQube was for local instances and SonarCloud should be used for online (clould) instances. Installing SonarQube on a Azure app (as suggested here: Combining SonarQube and Azure DevOps - Microsoft Playground feels wrong and costs to have the app running.

The only thing that comes to mind is creating a script to run the sonarQube analysis and only insert a job to run the script in the pipeline. I did not try this yet. The question here is if it is then still possible to get the results back to devops.

My questions

  1. Is the setup that I have in mind possible?
  2. What do I need to do in order to make it happen?
2

Hello @NickVdB,

Yes what you have in mind is possible. Many things are doable with SonarQube (SQ) :slight_smile:

Just to clarify between us (and potential other readers), by Azure DevOps (ADO) you refer to the cloud service https://dev.azure.com, not Azure DevOps Server (self-hosted).

To start, I want to make sure you make the difference between 2 important things:

  • the analysis part, when a scanner is executed. Virtually, this can run anywhere and target any SQ which has access to Internet. So there is no compatibility concern here between SQ and ADO.
  • the Pull-Request decoration part, where SQ sends its report to your ALM so you can have results here directly. Here indeed not all combination works, as you can read on the PR doc. SQ is only supported on ADO Server.

You can perfectly have the first one but work without the second. You will have analysis results in SQ only, not sent to ADO (by what we call Pull Request Decoration). But worry not, this in the pipe: MMF-1856: SonarQube integrates with Cloud ALMs.

To focus on having a working code analysis here is how to proceed:

  • as you mentioned developing on VisualStudio, I assume your code is built with MSBuild. The theory would say that you have to use our Integrate with MSBuild option. But you cannot, as analyzing C/C++ requires to wrap your build call with build-wrapper. In our doc SonarScanner for Azure DevOps, you will find a paragraph Analyzing a C/C++/Obj-C project which explains it all.
  • about the Service-connection, the agent that will run the analysis (your Win10 machine) needs this access (to load plugins, rules, etc. then send the analysis results), not ADO directly. So if I understood correctly you installed SQ also on this Win10 server, so even 127.0.0.1:9000 should work (of course for a production setup it will be different: SQ on another server, ideally a DNS to reach to it, etc.)
  • about the error you have:
    [error]No agent found in pool which satisfies the following demand: java
    I’m not sure I can’t help a lot here. I guess it has something to do with how your agent is confugured in ADO. Have a look at their capabilities doc to make sure this is well setup. You also try to run the scanner from the agent itself, to make sure it works well.

I hope this will help.

1 Like