We are having this problem with a private project on Laravel that we are working at the moment.
It seems that we might need some more configuration to allow SonarCloud testing our code.
As you can see in the picture, there are also New Code Smells that are retrieved from the test files.
After some research through SonarCloud Docs and Internet on code coverage with PHPUnit, not sure how what we need to apply to make it work.
Does anybody can tell me what’s happening or what else we have to do?
I have a couple question to help investigate what’s going on here:
Are you using Autoscan to analyze your PHP project? You should know that coverage is not supported with Autoscan currently.
Did you set the sonar.php.tests.reportPath setting correctly pointing to your coverage reports file?
Are those report files correctly generated before you run the SonarCloud analysis?
Do you see anything interesting in the scanner logs related to coverage? How about if you run in debug mode? It should at least show the coverage report files that were processed, possibly some errors if something went wrong with the report processing.
Regarding the code smells reported on test files, it’s normal behavior, they should have the same quality as the rest of your source files.
Thanks for your welcome and your lines. And sorry for my newbie questions but it’s my first time using SonarCloud.
Regarding the Autoscan feature, I did not do the setup for our projects but would say that is enabled, yes. As PR and the code from the repository it’s been scanned.
We don’t have any .sonarcloud.properties file on our repository, so that would be a next step to do I guess, and setup sonar.php.tests.reportPath.
Not sure which settings are mandatory on .sonarcloud.properties, there’s any guide about it?
On my Localhost, tests are running returning an OK message and coverage reports are correctly generated:
Thanks for sharing your yaml file. Seems like you are missing the sonar.php.coverage.reportPath setting in the EXTRA_ARGS of your pipe’s configuration.
We don’t see your full pipeline here so maybe it’s already the case but if you want the coverage to be correctly processed it needs to be computed as part of your bitbucket pipeline so that the scanner can access the generate files.
image: williamyeh/ansible:ubuntu18.04
pipelines:
custom: ... # Used to deploy on DEV not related to quality gate on SonarCloud
pull-requests: ... # SonarCloud check quality code and coverage
Added sonar.php.coverage.reportPath to bitbucket-pipelines.yml:
No need to add vendor/bin/phpunit so Bitbucket Pipelines can run the tests and get the reports?
Just saying cause my test reports are not pushed to Bitbucket repository.
Sure, here you have the logs and I saw this inbetween:
No PHPUnit test report provided (see 'sonar.php.tests.reportPath' property)
No PHPUnit coverage reports provided (see 'sonar.php.coverage.reportPaths' property)
Do I need to publish my reports to the repository to make it work or the scanner will run the tests and get the generated reports?
Changed sonar.php.coverage.reportPath to sonar.php.coverage.reportPaths and Added -Dsonar.php.tests.reportPath=\"codeCoverage/test.xml\" but still getting the same results.
As you can see, I’m loading an image that has phpunit and Xdebug (because with other images was throwing an Error: No code coverage driver is available while doing the vendor/bin/phpunit).
With this image the Bitbucket Pipeline did run vendor/bin/phpunit but with really different results as in my Localhost, check it out:
Seeing the result of your unit tests running in the pipeline, I’m wondering if you tests would be relying on some external service ? Because I see that 5 of your tests are in failure because they received a code 500 instead of 200, it looks like http status code ?
If your unit tests are relying on an external service to pass, then you should:
make this service public somehow, so that the machine that run your unit tests from bitbucket pipeline infrastructure can access it (I guess it could be complicated and we can’t provide help with that)
or rewrite your tests so that they don’t rely on any external services (using mocks for exemple) and so that they can run anywhere
And I’m working on a way to mock Guzzle Client for my tests, as the app is calling an external API using Guzzle, so the tests can’t reach the API, that is not public.
What I wanted to confirm with you is, that sonar-project.properties is useless for Bitbucket as Aurélie said, so maybe I could maybe skip EXTRA_ARGS: '-Dproject.settings=sonar-project.properties' in my bitbucket-pipelines.yml.
My bitbucket-pipelines.yml looks like this at the moment:
image: williamyeh/ansible:ubuntu18.04
pipelines:
pull-requests:
'**':
- step:
name: Run PHPUnit tests
image: pawelgradziel/php73-cli-xdebug-composer:latest
caches:
- composer
script:
- composer install --no-progress --no-suggest
- vendor/bin/phpunit --log-junit ./test-reports/phpunit.junit.xml --coverage-clover ./test-reports/phpunit.coverage.xml
artifacts: # defining the artifacts to be passed to each future step.
- test-reports/phpunit**
- step: &step-sonarcloud-scan
name: Analyze code on SonarCloud
script:
- pipe: sonarsource/sonarcloud-scan:0.1.5
variables:
SONAR_TOKEN: ${SONAR_TOKEN}
EXTRA_ARGS: '-Dproject.settings=sonar-project.properties'
I think she said it’s the .sonarcloud.properties file that is currently useless on Bitbucket.
The sonar-project.properties file can still be useful if you have a lot of configuration inside, otherwise if you just have a few you can pass them directly in the EXTRA_ARGS variable of your pipeline like this for exemple: EXTRA_ARGS: '-Dsonar.sources=src -Dsonar.tests=src'
One thing you need to be careful about are the paths in the clover.xml. I had the same issue when I send clover generated by PHPUnit (on WordPress project, but that shouldn’t matter).
The clover.xml that is generated will create file name attribute that is absolute on your system - since I generated my tests locally, the name attribute was something like
So I usually do a search-replace on my clover file so that only wp-content/plugins/... part is left (I delete the first part). Once I send this to SQ in my Jenkins pipeline, the coverage will be shown - not the same as in PHPUnit report, but it will be shown (my guess is that SQ can then generate the coverage internally somehow).
