Sonar scanner fails to load Go test results

[rayharris]

SCM: github.com
CI: travis
Scanner: travis addon sonar scanner
Language: go 1.13.x

sonar-project.properties

sonar.projectKey=my-project-key
sonar.projectName=my-project-name
sonar.exclusions=**/*_test.go,**/*_generated*.go,**/*_generated/**,**/vendor/**
sonar.test.inclusions=**/*_test.go
sonar.test.exclusions=**/*_generated*.go,**/*_generated/**,**/vendor/**
sonar.go.tests.reportPaths=report.json
sonar.go.coverage.reportPaths=coverage.out

Snippet from .travis.yml

    - stage: sonar
      name: "Run SonarCloud Analysis"
      script:
        - go test -coverprofile=coverage.out ./...
        - go test -json ./... > report.json
        - cat report.json
        - sonar-scanner

The first two lines in the script run successfully and the code coverage report is loaded successfully.

I uploaded the report.json output as report.txt:

report.txt (94.4 KB)

This is the first few lines of the output of sonar-scanner:

$ sonar-scanner
INFO: Scanner configuration file: /home/travis/.sonarscanner/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/travis/gopath/src/github.com/open-cluster-management/multicloud-operators-cluster-controller/sonar-project.properties
INFO: SonarQube Scanner 3.0.3.778
INFO: Java 1.8.0_151 Oracle Corporation (64-bit)
INFO: Linux 4.4.0-101-generic amd64
INFO: User cache: /home/travis/.sonar/cache
INFO: SonarQube server 8.0.0
INFO: Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=689ms
INFO: Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
INFO: User cache: /home/travis/.sonar/cache

These are the lines that show the output for the Go analysis which indicates that it can’t load the test execution report:

INFO: Sensor SonarGo [go]
INFO: 35 source files to be analyzed
INFO: 35/35 source files have been analyzed
INFO: Sensor SonarGo [go] (done) | time=967ms
INFO: Sensor Go Unit Test Report [go]
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/clusterimport and test Test_bootstrapServiceAccountNsN/nil_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/clusterimport and test Test_bootstrapServiceAccountNsN/empty_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/clusterimport and test Test_bootstrapServiceAccountNsN/good_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/clusterimport and test Test_bootstrapServiceAccountNsN
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile/ClusterDeployment_DNE
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile/Only_ClusterDeployment
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile/ClusterDeployment_&_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile/ClusterDeployment_&_EndpointConfig_with_ImagePullSecret
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_clusterRegistryNsN/null_cluster_deployment
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_clusterRegistryNsN/empty_cluster_deployment
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_clusterRegistryNsN/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_clusterRegistryNsN
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_getClusterRegistryCluster/empty_clusterDeployment
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_getClusterRegistryCluster/client_error
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_getClusterRegistryCluster/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_getClusterRegistryCluster
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_newClusterRegistryCluster/empty_clusterDeployment
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_newClusterRegistryCluster/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_newClusterRegistryCluster
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_createClusterRegistryCluster/nil_clusterDeployment
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_createClusterRegistryCluster/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_createClusterRegistryCluster
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_bootstrapServiceAccountNsN/nil_cluster
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_bootstrapServiceAccountNsN/empty_cluster.Name
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_bootstrapServiceAccountNsN/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_bootstrapServiceAccountNsN
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_NewBootstrapServiceAccount/nil_cluster
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_NewBootstrapServiceAccount/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_NewBootstrapServiceAccount
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_getBootstrapServiceAccount/empty_cluster_registry_object
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_getBootstrapServiceAccount/SA_does_not_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_getBootstrapServiceAccount/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_getBootstrapServiceAccount
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_createBootstrapServiceAccount/SA_already_exists
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_createBootstrapServiceAccount/empty_cluster_registry_object
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_createBootstrapServiceAccount/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterregistry and test Test_createBootstrapServiceAccount
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterRegistryNsN/nil_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterRegistryNsN/empty_EndpointConfig.Spec.ClusterName
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterRegistryNsN/empty_EndpointConfig.Spec.ClusterNamespace
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterRegistryNsN/no_error
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterRegistryNsN
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getClusterRegistryCluster/nil_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getClusterRegistryCluster/cluster_does_not_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getClusterRegistryCluster/cluster_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getClusterRegistryCluster
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterReconcileMapper_Map/green
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_clusterReconcileMapper_Map
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile/endpointConfig_do_not_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile/terminating_endpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile/invalid_endpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile/cluster_does_not_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile/missing_resource_to_generate_secret
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile/success
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test TestReconcileEndpointConfig_Reconcile
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_importSecretNsN/nil_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_importSecretNsN/empty_EndpointConfig.Spec.ClusterName
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_importSecretNsN/empty_EndpointConfig.Spec.ClusterNamespace
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_importSecretNsN/no_error
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_importSecretNsN
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getImportSecret/nil_EndpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getImportSecret/secret_does_not_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getImportSecret/secret_does_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_getImportSecret
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_newImportSecret/nil_scheme
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_newImportSecret/nil_endpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_newImportSecret/empty_endpointConfig
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_newImportSecret/no_error
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_newImportSecret
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_createImportSecret/no_error
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_createImportSecret/secret_already_exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_createImportSecret
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/endpointconfig and test Test_toYAML
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestUniqueStringSlice
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestRemoveFromStringSlice
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestAppendIfDNE
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestFileExist/exist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestFileExist/dne
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestFileExist
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestAddFinalizer/add
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestAddFinalizer/don't_add
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestAddFinalizer
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestRemoveFinalizer/don't_remove
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestRemoveFinalizer/remove
WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/utils and test TestRemoveFinalizer
INFO: Sensor Go Unit Test Report [go] (done) | time=71ms
INFO: Sensor Go Cover sensor for Go coverage [go]
INFO: Load coverage report from '/home/travis/gopath/src/github.com/open-cluster-management/multicloud-operators-cluster-controller/coverage.out'
INFO: Sensor Go Cover sensor for Go coverage [go] (done) | time=112ms

[CameronG]

Full disclose I don’t use travis and I’m not sure why you’re seeing those warnings about missing test files, but I have been fighting with getting Go coverage and test reports into Sonar for a few days now and finally have it working, so here is a link to the rundown of my build-test process and settings. My inclusions and exclusions are a bit different from yours and I recall seeing some posts about how sonar scanner uses the exclusions and test.exclusions that can result in some behavior that is not intuitive from the property names. Take a look at and maybe emulate my config below and see if that helps?

SonarGo code coverage 0%

For anyone else who may find it helpful, now that my build process is working, here is a simplified run down of my settings and build process: My first step is to run my tests and generate my a coverage report and test report: go test "./..." -coverprofile="coverage.out" -covermode=count -json > report.json; I also added an optional step to generate a go linter report using this 3rd party cli golangci-lint run ./... --verbose --no-config --out-format checkstyle > golangci-lint.out My sonar.…

[rayharris]

Cameron, is your Go project using modules? That’s what seems to be the issue.

Just to check, I trimmed my sonar-project.properties file to just be

sonar.projectKey=my-project-key
sonar.projectName=my-project-name
sonar.go.tests.reportPaths=report.json
sonar.go.coverage.reportPaths=coverage.out

to makes sure no files were excluded and it’s still not able to load the test results:

18:29:04.518 WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile/ClusterDeployment_&_EndpointConfig_with_ImagePullSecret
18:29:04.519 WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test TestReconcileClusterDeployment_Reconcile
18:29:04.519 WARN: Failed to find test file for package github.com/open-cluster-management/multicloud-operators-cluster-controller/pkg/controller/clusterdeployment and test Test_clusterRegistryNsN/null_cluster_deployment

[rayharris]

So I figured it out. The secret was in Cameron’s properties file, but it’s not clear what it is.

When I ran with just this (omitting projectKey and projectName):

sonar.go.tests.reportPaths=report.json
sonar.go.coverage.reportPaths=coverage.out

I got code coverage, but not test results. So my assumption was that the default values for sonar.sources and sonar.tests was correct and the issue was in how tests were processed. This assumption turned out to be wrong (as they generally are).

I cloned the Sonar Slang repo at https://github.com/SonarSource/slang, added additional debugging output, and built and ran the sonar-go plugin locally. What it showed was that no test files were being picked up, even with sonar.test.inclusions being specified.

I added the sonar.tests=. line to the properties file and test results started being processed, but then code coverage wasn’t being reported. Finally I added sonar.sources=. and both code coverage and tests results showed up in Sonar Cloud.

The best I can figure is that sonar.sources was defaulting correctly, but sonar.tests wasn’t. When I added sonar.tests=. that then broke the default value for sonar.sources and both have to be supplied even if you’re just using the default values. Why is it like this? I’m not that good at Java, so I don’t know.

Here’s my final sonar-project.properties file:

sonar.projectKey=my-project-key
sonar.projectName=my-project-name
sonar.sources=.
sonar.exclusions=**/*_test.go,**/*_generated*.go,**/*_generated/**,**/vendor/**
sonar.tests=.
sonar.test.inclusions=**/*_test.go
sonar.test.exclusions=**/*_generated*.go,**/*_generated/**,**/vendor/**
sonar.go.tests.reportPaths=report.json
sonar.go.coverage.reportPaths=coverage.out

So Cameron had it right by including the sonar.sources and sonar.tests properties, but I was leaving them out thinking they would default correctly when they didn’t.

[chrisinmtown (Chris Lott)]

I found a workaround to report coverage of my module. I do not expect this to work for third-party modules. My module is maintained in a subdirectory. I added a build-script step to rewrite the paths in the coverage data file, then the SonarScanner matched the scanned path with the reported path and coverage data appeared in SonarCloud.io. Sed to the rescue:

sed -i -e 's/^themodule/TheModSubdir/' coverage.out

HTH

[Bui_Nguyen_Ly (Bùi Nguyên Lý)]

I use sonar.exclusions instead of sonar.go.exclusions and everything works ok.