Sonarqube Scanner ignores my Coverage report "test-result-codecoverage.json"

Truong_Phan · September 8, 2020, 6:22am

Hi

We have CI/CD process by using Atlassian’s Bamboo where we have a Bamboo’s task to run Salesforce SFDX CLi to capture Apex Test Code Coverage report in the JSON format with a correct file name ( test-result-codecoverage.json ). This Apex Test Code Coverage report (test-result-codecoverage.json ) is fed into the SonarQube Scanner via the parameter ( -Dsonar.apex.coverage.reportPath=/workspace/project/SeedNSync/config/sonarqube/test-result-codecoverage.json ).
However we don’t see SonarQube Dashboard populates the Test Code Coverage data.

Our environment:

SonarQube v6.7
Scanner v2.8
SFDX CLI v7.12.3

Our Commands:

** Note: We have tried to provide a full path name with and without the Test Code Coverage file name for the Sonarqube’s variable (** sonar.apex.coverage.reportPath **) but it still not working.

## SFDX CLI Test Run Command: 
sfdx force:apex:test:run -l RunSpecifiedTests -s CDM_Test_Classes -c -r json -d /workspace -u cicd_pipeline@team.company.com -w 30'

## SonarQube Scanner command:
  /opt/sonar-scanner-2.8/bin/sonar-scanner --debug \
  -Dsonar.projectKey="$sonarKey" \
  -Dsonar.sources=/workspace/project/SeedNSync/force-app/main/default/classes \
  -Dsonar.projectVersion=1.1 \
  -Dsonar.projectName=b2c-cdm-vs \
  -Dsonar.host.url=${bamboo.sonarqube67_host_url} \
  -Dsonar.login=${bamboo.sonarqube67_n108203_auth_token} \
  -Dsonar.apex.coverage.reportPath=/workspace/project/SeedNSync/config/sonarqube/test-result-codecoverage.json \
  -Dsonar.verbose=true \
  -Dsonar.exclusions=MetadataService.cls \

Our Bamboo Log

. . .
build	28-Aug-2020 02:32:10	{
build	28-Aug-2020 02:32:10	  "status": 0,
build	28-Aug-2020 02:32:10	  "result": {
build	28-Aug-2020 02:32:10	    "summary": {
build	28-Aug-2020 02:32:10	      "outcome": "Passed",
build	28-Aug-2020 02:32:10	      "testsRan": 150,
build	28-Aug-2020 02:32:10	      "passing": 150,
build	28-Aug-2020 02:32:10	      "failing": 0,
build	28-Aug-2020 02:32:10	      "skipped": 0,
build	28-Aug-2020 02:32:10	      "passRate": "100%",
build	28-Aug-2020 02:32:10	      "failRate": "0%",
build	28-Aug-2020 02:32:10	      "testStartTime": "Aug 28, 2020 12:23 PM",
build	28-Aug-2020 02:32:10	      "testExecutionTime": "737936 ms",
build	28-Aug-2020 02:32:10	      "testTotalTime": "697521 ms",
build	28-Aug-2020 02:32:10	      "commandTime": "448077 ms",
build	28-Aug-2020 02:32:10	      "hostname": "https://*****.my.salesforce.com",
build	28-Aug-2020 02:32:10	      "orgId": "00D2N0000009HPGUA2",
build	28-Aug-2020 02:32:10	      "username": "***cicd_pipeline@team.company.com",
build	28-Aug-2020 02:32:10	      "testRunId": "7072N00000Dw6ly",
build	28-Aug-2020 02:32:10	      "userId": "0052P000000ILSbQAO",
build	28-Aug-2020 02:32:10	      "testRunCoverage": "69%",
build	28-Aug-2020 02:32:10	      "orgWideCoverage": "81%"
build	28-Aug-2020 02:32:10	    },

...

build	28-Aug-2020 02:32:13	          "TestMethodName": "testLookupField",
build	28-Aug-2020 02:32:13	          "NumLinesCovered": 53,
build	28-Aug-2020 02:32:13	          "ApexClassOrTrigger": {
build	28-Aug-2020 02:32:13	            "attributes": {
build	28-Aug-2020 02:32:13	              "type": "Name",
build	28-Aug-2020 02:32:13	              "url": "/services/data/v46.0/tooling/sobjects/ApexClass/01p5L0000008XPSQA2"
build	28-Aug-2020 02:32:13	            },
build	28-Aug-2020 02:32:13	            "Id": "01p5L0000008XPSQA2",
build	28-Aug-2020 02:32:13	            "Name": "lookupfieldController"
build	28-Aug-2020 02:32:13	          },
build	28-Aug-2020 02:32:13	          "NumLinesUncovered": 9
build	28-Aug-2020 02:32:13	        }
build	28-Aug-2020 02:32:13	      ],
build	28-Aug-2020 02:32:13	      "summary": {
build	28-Aug-2020 02:32:13	        "totalLines": 60310,
build	28-Aug-2020 02:32:13	        "coveredLines": 41982,
build	28-Aug-2020 02:32:13	        "testRunCoverage": "69%",
build	28-Aug-2020 02:32:13	        "orgWideCoverage": "81%"
build	28-Aug-2020 02:32:13	      }
build	28-Aug-2020 02:32:13	    }
build	28-Aug-2020 02:32:13	  }
build	28-Aug-2020 02:32:13	}
build	28-Aug-2020 02:32:13	

Environment variables:
build	28-Aug-2020 02:32:13	HOSTNAME=f768342d173b
build	28-Aug-2020 02:32:13	YARN_VERSION=1.22.0
build	28-Aug-2020 02:32:13	PWD=/workspace
build	28-Aug-2020 02:32:13	HOME=/root
build	28-Aug-2020 02:32:13	SFDX_AUTOUPDATE_DISABLE=true
build	28-Aug-2020 02:32:13	SONAR_RUNNER_HOME=sonar-scanner
build	28-Aug-2020 02:32:13	SFDX_CLI_VERSION=7.12.3
build	28-Aug-2020 02:32:13	NO_PROXY=
build	28-Aug-2020 02:32:13	SHLVL=2
build	28-Aug-2020 02:32:13	HTTPS_PROXY=
build	28-Aug-2020 02:32:13	HTTP_PROXY=
build	28-Aug-2020 02:32:13	VBT_VERSION=v1.11.3
build	28-Aug-2020 02:32:13	PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/sonar-scanner-2.8/bin
build	28-Aug-2020 02:32:13	NODE_VERSION=12.16.1
build	28-Aug-2020 02:32:13	_=/usr/bin/env


error	28-Aug-2020 02:32:15	+ echo
error	28-Aug-2020 02:32:15	++ cat test-run-id.txt
build	28-Aug-2020 02:32:15	TEST_RUN_ID: 7072N00000Dw6ly
error	28-Aug-2020 02:32:15	+ echo TEST_RUN_ID: 7072N00000Dw6ly
error	28-Aug-2020 02:32:15	++ cat test-run-id.txt
error	28-Aug-2020 02:32:15	+ export TEST_RUN_ID=7072N00000Dw6ly
build	28-Aug-2020 02:32:15	
error	28-Aug-2020 02:32:15	+ TEST_RUN_ID=7072N00000Dw6ly
error	28-Aug-2020 02:32:15	+ echo
error	28-Aug-2020 02:32:15	++ id
build	28-Aug-2020 02:32:15	WHOAMI: uid=134(ci) gid=1003(ci) groups=1003(ci),10(wheel),994(docker) context=system_u:system_r:initrc_t:s0
error	28-Aug-2020 02:32:15	+ echo WHOAMI: 'uid=134(ci)' 'gid=1003(ci)' 'groups=1003(ci),10(wheel),994(docker)' context=system_u:system_r:initrc_t:s0
build	28-Aug-2020 02:32:15	Current PWD:
error	28-Aug-2020 02:32:15	+ echo Current PWD:
build	28-Aug-2020 02:32:15	/opt/bamboo_agent/xml-data/build-dir/B2CSAL-B2CCDMSFSEC9-JOB1
error	28-Aug-2020 02:32:15	+ pwd
error	28-Aug-2020 02:32:15	+ ls -al
build	28-Aug-2020 02:32:15	total 32680
build	28-Aug-2020 02:32:15	drwxr-xr-x.  6 ci   ci        222 Aug 28 02:32 .
build	28-Aug-2020 02:32:15	drwxr-xr-x.  7 ci   ci        148 Aug 28 02:22 ..
build	28-Aug-2020 02:32:15	drwxr-xr-x.  7 ci   ci        186 Aug 28 02:22 codequality
build	28-Aug-2020 02:32:15	drwxr-xr-x.  4 ci   ci         65 Aug 28 02:22 config
build	28-Aug-2020 02:32:15	drwxr-xr-x.  5 ci   ci         48 Aug 28 02:22 project
build	28-Aug-2020 02:32:15	drwxr-xr-x. 10 ci   ci        229 Aug 28 02:22 sfdx-build-tool
build	28-Aug-2020 02:32:15	-rw-r--r--.  1 root root 31980392 Aug 28 02:32 test-result-7072N00000Dw6ly.json
build	28-Aug-2020 02:32:15	-rw-r--r--.  1 root root    18831 Aug 28 02:32 test-result-7072N00000Dw6ly-junit.xml
build	28-Aug-2020 02:32:15	-rw-r--r--.  1 root root  1457674 Aug 28 02:32 test-result-codecoverage.json
build	28-Aug-2020 02:32:15	-rw-r--r--.  1 root root       15 Aug 28 02:23 test-run-id.txt

Cameron · September 8, 2020, 12:37pm

Hi @Truong_Phan,

Welcome to the community!

Can I check one thing. Based on this:

Are you using the SonarSource Apex analyzer? Because it was only released with SonarQube 7.5 and AFAIK it’s not compatible with SonarQube 6.7.

Regards,

Cameron.

Truong_Phan · September 9, 2020, 12:42am

Hi @Cameron,

Thanks.
I think we are using CodeScan Apex Analyzer v 4.2.2

INFO: Execute CodeScan Apex Analysis 4.2.2

Truong_Phan · September 9, 2020, 12:46am

Hi @Cameron,

Could you please point me to the official document where it states that CodeScan Apex Analyzer is not compatible with SonarQube 6.7?
However the CodeScan Apex Analyzer is working on other components of Apex. Except it won’t pick-up the Code Coverage Data.

Regards,
Truong

Truong_Phan · September 9, 2020, 12:59am

Hi @Cameron,

Thanks for your response which has help me to clear my mind to drill down a bit further on Sonarqube’s documentations.
I can see that Sonarqube’s 7.7 onward did mention about the parameter sonar.apex.coverage.reportPath.

Regards,
Truong

Cameron · September 9, 2020, 6:55am

Hi @Truong_Phan,

In fact the SonarSource Apex analyzer I referred to is not the same as the CodeScan Apex Analyzer, which is written by CodeScan, a third party that isn’t associated with us. So for your current issues with coverage on SonarQube 6.7 you should discuss with the team at CodeScan. And of course if you’d like to try out the SonarSource-written scanner and you have SonarQube Enterprise Edition, you’re welcome to upgrade to the 7.9 LTS or a later version

Regards,

Cameron.

Sourav_Neogi · August 6, 2021, 2:54am

Hi @Cameron ,

I have the same issue.We are using Jenkins as CI tool. Although I am using SonarSource Apex analyzer. Our test code coverage isn’t getting populated and showing 0% in SonarQube although using the sfdx command we found the coverage to be 89%. I am new to this too, so could you please guide me on this?

Also when i run the command sfdx force:apex:test:run -c -r json -l RunLocalTests -w 60 -d /testCov -u $(CodeCoverageUser)
I am unable to view any newly formed test-result-codecoverage.json file on my local workspace.

Cameron · August 9, 2021, 9:11am

Hi @Sourav_Neogi,

Welcome to the Community!

Can you confirm that you’ve set sonar.apex.coverage.reportPath to point to the test-result-codecoverage.json file? Do you have an extract from the SonarQube scanner logs that show the file being picked up (or not) by the SonarQube scan?

Regards,

Cameron.

Marco_Rodriguez · August 5, 2022, 12:50am

Hi Cameron - Hoping you can help to confirm this because I am unable to find this in sonarqube documentation.

This is our situation:
We have Sonarqube Community Edition which does NOT support Apex (Salesforce language), this is fine but we want to report the code coverage obtained from Salesforce to Sonarqube, we try to do this through the sonar.apex.coverge.reportPath parameter but it is ignored.

Here is my question:
Should the sonar.apex.coverge.reportPath parameter work and import the Salesforce-generated apex coverage file in Sonarqube Community Edition 8.4.2 OR is Sonarqube Enterprise Edition required for the sonar.apex.coverge.reportPath parameter to work?