ERROR: sonar.exclusions property not recognising * or ** values for windows machine

jenkins
sonarqube

(akshay) #1

SonarQube server Version: 7.1 (build 11001)
SonarQube Scanner for Jenkins: 2.6.1
Jenkins version: 2.41

This is my directory structure:

E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/

    E:
    |-- Jenkins
    |    |--workspace
    |    |   |--dev_A
    │    |   |   |--src   
    │    |   |   |   |--github.com
    │    │   |   |   |   |--ABC
    │    │   |   |   |   |  |--A
    │    │   |   |   |   |  |  |--X
    │    │   |   |   |   |  |  |--Y
    │    │   |   |   |   |  |  |--vendor
    |    |   |   |   |   |  |  |--static-analysis.xml

this is my jenkins job that I had written inorder to exclude vendor directory:

    sonar.projectBaseDir="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/"
    sonar.sourceEncoding="UTF-8"
    sonar.projectKey=A
    sonar.projectName=A
    sonar.projectVersion=${BUILD_NUMBER}
    sonar.golint.reportPath="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/static-analysis.xml"
    sonar.coverage.reportPath="coverage.xml"
    sonar.exclusions= **/vendor/**
    sonar.sources=.
    sonar.sources.inclusions=**/*.go*

When I executed the job I got the following error:

ERROR: Error during SonarQube Scanner execution  
ERROR: Illegal char <*> at index 0: **/vendor/**

when I tried rectifying the error according to the official sonar docs by implementing:

sonar.exclusions= **/vendor/**/*

I got this error:

ERROR: Error during SonarQube Scanner execution
ERROR: Illegal char <*> at index 0: **/vendor/**/*

any help on this issue would really be appreciated.


(G Ann Campbell) #2

Hi,

Before we get to your exclusions, I’d like to take a look at your other settings.

First, your projectBaseDir is by default the directory from which analysis is launched. It’s rare that you actually need to set this. I’m going to guess that analysis would otherwise launch from the workspace directory and you’re using projectBaseDir to drill into that directory so that when you set sonar.sources to ., it works. You’re going around your ear to scratch your elbow.

Instead, eliminate sonar.projectBaseDir and simply set sonar.sources to dev_A/src/github.com/ABC/A/. Or… you could set sonar.sources=dev_A/src/github.comABC/A/X,dev_A/src/github.comABC/A/Y,dev_A/src/github.comABC/A/static-analysis.xml and eliminate the need to exclude vendor.

As for your inclusion, it looks like you’re trying to restrict analysis to a single language. I’m curious why.

 
Ann


(akshay) #3

Hi Ann,

truly appreciated your pro-active response on the mentioned blocker which is an impediment on work execution.

Well, you wanted to know about why only one language is targeted. Most recently we have started with sonar and initially just targeting Golang microservices which contains the 3rd party packages, which may include .js, .jsx, .xml and .json files. I believe to analyze technical debt & its better to target
own source code instead 3rd party as, we can continuously improve our quality of code.

However, I have implemented your suggestion but its hard luck and really its very tough time as I have tried more than 50+ permutation and combination to get the success on this area but all FAILED. :slight_smile:

This is the jenkins job modified according to your suggestions:

    sonar.sourceEncoding="UTF-8"
    sonar.projectKey=A
    sonar.projectName=A
    sonar.projectVersion=${BUILD_NUMBER}
    sonar.golint.reportPath="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/static-analysis.xml"
    sonar.coverage.reportPath="coverage.xml"
    sonar.sources= "dev_A/src/github.comABC/A/X, dev_A/src/github.comABC/A/Y, dev_A/src/github.comABC/A/static-analysis.xml"
    sonar.sources.inclusions=**/*.go*

Under jenkins job logs it is still showing that vendor directory is invoked. Please do consider the following logs:

INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\ABC\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\ABC\godep-DATA-DOG-go-sqlmock\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\ABC\godep-DATA-DOG-go-sqlmock\examples\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\ABC\godep-DATA-DOG-go-sqlmock\examples\basic\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\ABC\godep-DATA-DOG-go-sqlmock\examples\blog\coverage.xml
INFO: no coverage file in package E:\Jenkins\workspace\dev_A\src\github.com\ABC\A\vendor\github.com\ABC\godep-DATA-DOG-go-sqlmock\examples\orders\coverage.xml

your further suggestions/solutions will definately give a kick start to this area. Your time and efforts are truly appreciated

Thanks a ton.


(G Ann Campbell) #4

Hi,

Does the vendor directory actually show up in the project’s code page in SonarQube? Also, could you share your analysis command? And finally, keeping 3rd-party code out of your analysis is what I would use exclusions for. But I would set them via the UI.

 
Ann


(akshay) #5

Hi Ann,

As you requested, here are the following things:

  1. Have a look at the codescan page:

         vendor/github.com/ABC/godep-DATA-DOG-go-sqlmock
         vendor/github.com/ABC/godep-DATA-DOG-go-sqlmock/examples
         vendor/github.com/ABC/godep-DATA-DOG-go-sqlmock/examples/basic
         vendor/github.com/ABC/godep-DATA-DOG-go-sqlmock/examples/blog
         vendor/github.com/ABC/godep-DATA-DOG-go-sqlmock/examples/orders
         vendor/github.com/ABC/godep-kisielk-sqlstruct.git
         vendor/github.com/denisenkom/go-mssqldb
         vendor/github.com/denisenkom/go-mssqldb/batch
         vendor/github.com/denisenkom/go-mssqldb/examples/bulk
         vendor/github.com/denisenkom/go-mssqldb/examples/simple
         vendor/github.com/denisenkom/go-mssqldb/examples/tsql
         vendor/github.com/golang/mock/gomock
         vendor/github.com/golang/mock/gomock/mock_matcher
         vendor/github.com/golang/mock/mockgen
         vendor/github.com/golang/mock/mockgen/model
         vendor/github.com/golang/mock/mockgen/tests/unexported_method
         vendor/github.com/golang/mock/sample
         vendor/github.com/golang/mock/sample/imp1
         vendor/github.com/golang/mock/sample/imp2
         vendor/github.com/golang/mock/sample/imp3
         vendor/github.com/golang/mock/sample/imp4
         vendor/github.com/golang/mock/sample/mock_user
         vendor/github.com/kjk/dailyrotate
         vendor/golang.org/x/crypto/acme
         vendor/golang.org/x/crypto/acme/autocert
         vendor/golang.org/x/crypto/bcrypt
         vendor/golang.org/x/crypto/blake2b
         vendor/golang.org/x/crypto/blake2s
         vendor/golang.org/x/crypto/blowfish
         vendor/golang.org/x/crypto/bn256
         vendor/golang.org/x/crypto/cast5
         vendor/golang.org/x/crypto/chacha20poly1305
         vendor/golang.org/x/crypto/chacha20poly1305/internal/chacha20
         vendor/golang.org/x/crypto/cryptobyte
         vendor/golang.org/x/crypto/cryptobyte/asn1
         vendor/golang.org/x/crypto/curve25519
         vendor/golang.org/x/crypto/ed25519
         vendor/golang.org/x/crypto/ed25519/internal/edwards25519
         vendor/golang.org/x/crypto/hkdf
         vendor/golang.org/x/crypto/md4
         vendor/golang.org/x/crypto/nacl/auth
         vendor/golang.org/x/crypto/nacl/box
         vendor/golang.org/x/crypto/nacl/secretbox
         vendor/golang.org/x/crypto/ocsp
         vendor/golang.org/x/crypto/openpgp
         vendor/golang.org/x/crypto/openpgp/armor
         vendor/golang.org/x/crypto/openpgp/clearsign
         vendor/golang.org/x/crypto/openpgp/elgamal
         vendor/golang.org/x/crypto/openpgp/errors
         vendor/golang.org/x/crypto/openpgp/packet
         vendor/golang.org/x/crypto/openpgp/s2k
         vendor/golang.org/x/crypto/otr
         vendor/golang.org/x/crypto/pbkdf2
         vendor/golang.org/x/crypto/pkcs12
         vendor/golang.org/x/crypto/pkcs12/internal/rc2
         vendor/golang.org/x/crypto/poly1305
         vendor/golang.org/x/crypto/ripemd160
         vendor/golang.org/x/crypto/salsa20
         vendor/golang.org/x/crypto/salsa20/salsa
         vendor/golang.org/x/crypto/scrypt
         vendor/golang.org/x/crypto/sha3
         vendor/golang.org/x/crypto/ssh
         vendor/golang.org/x/crypto/ssh/agent
         vendor/golang.org/x/crypto/ssh/knownhosts
         vendor/golang.org/x/crypto/ssh/terminal
         vendor/golang.org/x/crypto/ssh/test
         vendor/golang.org/x/crypto/ssh/testdata
         vendor/golang.org/x/crypto/tea
         vendor/golang.org/x/crypto/twofish
         vendor/golang.org/x/crypto/xtea
         vendor/golang.org/x/crypto/xts
         vendor/golang.org/x/net/bpf
         vendor/golang.org/x/net/context
         vendor/golang.org/x/net/context/ctxhttp
         vendor/golang.org/x/net/dict
         vendor/golang.org/x/net/dns/dnsmessage
         vendor/golang.org/x/net/html
         vendor/golang.org/x/net/html/atom
         vendor/golang.org/x/net/html/charset
         vendor/golang.org/x/net/http2
         vendor/golang.org/x/net/http2/h2demo
         vendor/golang.org/x/net/http2/h2i
         vendor/golang.org/x/net/http2/hpack
         vendor/golang.org/x/net/icmp
         vendor/golang.org/x/net/idna
         vendor/golang.org/x/net/internal/iana
         vendor/golang.org/x/net/internal/nettest
         vendor/golang.org/x/net/internal/socket
         vendor/golang.org/x/net/internal/timeseries
    
  2. This is the analysis command I am using to run the sonar scanner via jenkins plugin:

     sonar.projectBaseDir="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/"
     sonar.sourceEncoding="UTF-8"
     sonar.projectKey=A
     sonar.projectName=A
     sonar.projectVersion=${BUILD_NUMBER}
     sonar.golint.reportPath="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/static-analysis.xml"
     sonar.coverage.reportPath="coverage.xml"
     sonar.exclusions= **/vendor/**
     sonar.sources=.
     sonar.sources.exclusions="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/vendor"
     sonar.sources.inclusions=**/*.go*
    
  3. I also have set that property via UI and here it is:

  4. When I used sonar.exclusions the execution failed and when I used sonar.sources.exclusions it passed. So, can you please explain why is this happening?

Thanks


(G Ann Campbell) #6

Hi,

Could you clarify what you mean by “the codescan page”? To be clear, I was talking about your project’s Code page.

Selection_187

What your screenshot shows is that you’ve set up a coverage exclusion. I.e. you’ve said, not “don’t look at these files at all”, but instead “exclude these files from coverage calculations”. You want to scroll down to the Files section and set your exclusion there:
Selection_188

You’ll have to give me the error it’s failing with first. :slight_smile: Off-hand, I’m guessing that sonar.sources.exclusions is unknown and therefore ignored.

 
Ann


(akshay) #8

Hi ann,

  1. You guessed it right, the page which I sent you earlier was project’s code page only.

  2. As suggested by you we tried to implement the suggestion, but it is still failing.
    This is what I have implemented as Suggested By you.:slight_smile:

  1. This is the error again I am getting after implementing the above suggestion.

     ERROR: Error during SonarQube Scanner execution
     ERROR: Illegal char <*> at index 0: **/vendor**
    

Thanks.


(G Ann Campbell) #9

Hi,

Can you provide debug analysis logs (-Dsonar.log.level=debug), please?

 
Ann


(akshay) #10

Hi Ann,

Delighted to see your response and superior response time. Thank you so much for co-ordination.

I implemented the suggestion given by you in Jenkins which is as follows:

 sonar.projectBaseDir="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/"
 sonar.sourceEncoding="UTF-8"
 sonar.projectKey=A
 sonar.projectName=A
 sonar.projectVersion=${BUILD_NUMBER}
 sonar.golint.reportPath="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/static-analysis.xml"
 sonar.coverage.reportPath="coverage.xml"
 sonar.exclusions= **/vendor/**
 sonar.sources=.
 sonar.sources.exclusions="E:/Jenkins/workspace/dev_A/src/github.com/ABC/A/vendor"
 sonar.sources.inclusions=**/*.go*
 sonar.log.level=debug

This are the logs generated in Jenkins please have a look:
LOGS.txt (54.6 KB)

Thanks :slight_smile:


(G Ann Campbell) #11

Hi,

You’re still defining exclusions in your analysis properties, and this value overrides any set via the UI. You should remove the sonar.exclusions line; it’s redundant now that you’re defining this in the UI. Fully moving this value to the UI should also sidestep any questions about character set. But also, your screenshot shows that you’ve defined this exclusion globally. You should really move that to the project level.

That said, it’s odd that I don’t see anything about exclusions in your analysis log except where you’ve specified the property. Please try the changes I’ve just suggested and analyze again. Here is what I suggest:

 sonar.projectKey=A
 sonar.projectName=A
 sonar.projectVersion=1.0
 sonar.sources=.
 sonar.sourceEncoding="UTF-8"
 sonar.log.level=debug

You’ll notice I’ve removed several parameters. I suggest you get basic analysis working with your desired exclusion, and then you can add back in slowly.

As a side note, you really don’t want to set sonar.projectVersion to build number because that means a version event will be set on each and every analysis, exempting those analyses from housekeeping and potentially leading to database bloat.

 
Ann


(akshay) #12

Hi Ann,

As suggested by you I have just implemented the basic analysis with inclusion property and desired exclusion set via UI. Here are the logs:
SonarLogs.txt (50.9 KB)

But still I can see, it is including the vendor directory.
Please help with your valuable inputs to this. :slight_smile:

Thankyou