Code Coverage 0.0% on a Laravel project having PHPUnit to do testing

We are having this problem with a private project on Laravel that we are working at the moment.
It seems that we might need some more configuration to allow SonarCloud testing our code.

As you can see in the picture, there are also New Code Smells that are retrieved from the test files.

After some research through SonarCloud Docs and Internet on code coverage with PHPUnit, not sure how what we need to apply to make it work.

Does anybody can tell me what’s happening or what else we have to do?

Thanks in advance,
Oscar

Hi Oscar!

Welcome to the community!

I have a couple question to help investigate what’s going on here:

  • Are you using Autoscan to analyze your PHP project? You should know that coverage is not supported with Autoscan currently.
  • Did you set the sonar.php.tests.reportPath setting correctly pointing to your coverage reports file?
  • Are those report files correctly generated before you run the SonarCloud analysis?
  • Do you see anything interesting in the scanner logs related to coverage? How about if you run in debug mode? It should at least show the coverage report files that were processed, possibly some errors if something went wrong with the report processing.

Regarding the code smells reported on test files, it’s normal behavior, they should have the same quality as the rest of your source files.

Hi Grégoire!

Thanks for your welcome and your lines. And sorry for my newbie questions but it’s my first time using SonarCloud.

Regarding the Autoscan feature, I did not do the setup for our projects but would say that is enabled, yes. As PR and the code from the repository it’s been scanned.

We don’t have any .sonarcloud.properties file on our repository, so that would be a next step to do I guess, and setup sonar.php.tests.reportPath.

Not sure which settings are mandatory on .sonarcloud.properties, there’s any guide about it?

On my Localhost, tests are running returning an OK message and coverage reports are correctly generated:

Will create .sonarcloud.properties file on our repository and do the setup to see if that was it.

Thanks and any other feedback is very welcome.
Oscar

After the last analysis, still getting 0.0%…

Here’s more information from my setup in case is helpful.
Here the phpunit.xml settings to generate test reports:

<log type="coverage-html" target="./codeCoverage" lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-clover" target="./codeCoverage/coverage.xml"/>
<log type="coverage-php" target="./codeCoverage/coverage.serialized"/>
<log type="coverage-text" target="php://stdout" showUncoveredFiles="false"/>
<log type="junit" target="./codeCoverage/test.xml"/>
<log type="testdox-html" target="./codeCoverage/testdox.html"/>
<log type="testdox-text" target="./codeCoverage/testdox.txt"/>

And in the .sonarcloud.properties:

sonar.projectKey=MY_PROJECT_KEY
sonar.php.coverage.reportPath=codeCoverage/coverage.xml
sonar.php.tests.reportPath=codeCoverage/test.xml

Thanks,
Oscar

It’s normal if you don’t have coverage while using Autoscan:

So adding the paths to your coverage files in the .sonarcloud.properties won’t change the result.

You will need to use another CI to analyze your project if you want to have the coverage computed in SonarCloud, like Travis for exemple.

We are currently using Bitbucket and looks I can’t use Travis CI, that means we have to use Bitbucket Pipelines for that?

Can you give me any other hint?

I’m checking on the guides but running a bit lost at the moment.

Hi Oscar,

Autoscan is not available for Bitbucket right now. So you can remove the .sonarcloud.properties file that won’t have any effect.

How do you currently launch your analysis?

Thanks

Hi Aurélie,

Thanks for your reply.

We have this in our bitbucket-pipelines.yml:

  pull-requests:
    '**': 
      - step: 
          name: Sonar check quality code
          script: 
            - pipe: sonarsource/sonarcloud-scan:1.0.0
              variables:
                EXTRA_ARGS: -Dsonar.projectDescription=\"Project with sonarcloud-scan pipe\" -Dsonar.eslint.reportPaths=\"report.json\"
                SONAR_SCANNER_OPTS: -Xmx512m
                DEBUG: "false"

In my Localhost, I just execute vendor/bin/phpunit on my project’s path where tests are executed properly and reports created correctly.

Thanks for sharing your yaml file. Seems like you are missing the sonar.php.coverage.reportPath setting in the EXTRA_ARGS of your pipe’s configuration.

We don’t see your full pipeline here so maybe it’s already the case but if you want the coverage to be correctly processed it needs to be computed as part of your bitbucket pipeline so that the scanner can access the generate files.

Sorry for the cut off.

image: williamyeh/ansible:ubuntu18.04

pipelines:
  custom: ...   # Used to deploy on DEV not related to quality gate on SonarCloud
  pull-requests: ...   # SonarCloud check quality code and coverage

Added sonar.php.coverage.reportPath to bitbucket-pipelines.yml:

  pull-requests:
    '**':
      - step:
          name: Sonar check quality code
          script:
            - pipe: sonarsource/sonarcloud-scan:1.0.0
              variables:
                EXTRA_ARGS: -Dsonar.projectDescription=\"Project with sonarcloud-scan pipe\" -Dsonar.eslint.reportPaths=\"report.json\" -Dsonar.php.coverage.reportPath=\"codeCoverage/coverage.xml\"
                SONAR_SCANNER_OPTS: -Xmx512m
                DEBUG: "false"

No need to add vendor/bin/phpunit so Bitbucket Pipelines can run the tests and get the reports?
Just saying cause my test reports are not pushed to Bitbucket repository.

Could you launch the analysis in debug mode and share the results of the logs of the scan pipe?

Thanks,

Sure, here you have the logs and I saw this inbetween:

No PHPUnit test report provided (see 'sonar.php.tests.reportPath' property)

No PHPUnit coverage reports provided (see 'sonar.php.coverage.reportPaths' property)

Do I need to publish my reports to the repository to make it work or the scanner will run the tests and get the generated reports?

Changed sonar.php.coverage.reportPath to sonar.php.coverage.reportPaths and Added -Dsonar.php.tests.reportPath=\"codeCoverage/test.xml\" but still getting the same results.

Yes you need to run your unit tests and generate the coverage reports in the Bitbucket Pipelines, before starting the sonarcloud-scan.

Thanks for your reply Grégoire,

Tried many things and did some advance but still stucked with the pipelines.
Here’s my .yaml file:

image: williamyeh/ansible:ubuntu18.04

pipelines:
  custom: ...
  pull-requests:
    '**':
      - step:
          name: Run tests and Generate coverage reports
          image: epcallan/php7-testing-phpunit:7.2-phpunit7
          caches:
            - composer
          script:
            - composer install --no-progress --no-suggest
            - vendor/bin/phpunit
      - step:
          name: Sonar check quality code
          script:
            - pipe: sonarsource/sonarcloud-scan:1.0.0
              variables:
                EXTRA_ARGS: -Dsonar.projectDescription=\"Project with sonarcloud-scan pipe\" -Dsonar.eslint.reportPaths=\"report.json\" -Dsonar.php.coverage.reportPaths=\"codeCoverage/coverage.xml\" -Dsonar.php.tests.reportPath=\"codeCoverage/test.xml\"
                SONAR_SCANNER_OPTS: -Xmx512m
                DEBUG: "false"

As you can see, I’m loading an image that has phpunit and Xdebug (because with other images was throwing an Error: No code coverage driver is available while doing the vendor/bin/phpunit).

With this image the Bitbucket Pipeline did run vendor/bin/phpunit but with really different results as in my Localhost, check it out:

Localhost

Bitbucket Pipelines

Any other comments or suggestions?

Seeing the result of your unit tests running in the pipeline, I’m wondering if you tests would be relying on some external service ? Because I see that 5 of your tests are in failure because they received a code 500 instead of 200, it looks like http status code ?

If your unit tests are relying on an external service to pass, then you should:

  • make this service public somehow, so that the machine that run your unit tests from bitbucket pipeline infrastructure can access it (I guess it could be complicated and we can’t provide help with that)
  • or rewrite your tests so that they don’t rely on any external services (using mocks for exemple) and so that they can run anywhere

Hi Grégoire,

Yes, this project is using GuzzleHttpClient to perform API calls on an external project.

Regarding on your comments will see how to continue, as now not sure how to anymore.

Thanks for your support.
Oscar

One last thing that would like to clear up with you guys.

I could run my tests with Bitbucket Pipelines with same results as we saw before:


And I’m working on a way to mock Guzzle Client for my tests, as the app is calling an external API using Guzzle, so the tests can’t reach the API, that is not public.

What I wanted to confirm with you is, that sonar-project.properties is useless for Bitbucket as Aurélie said, so maybe I could maybe skip EXTRA_ARGS: '-Dproject.settings=sonar-project.properties' in my bitbucket-pipelines.yml.

My bitbucket-pipelines.yml looks like this at the moment:

image: williamyeh/ansible:ubuntu18.04

    pipelines:
      pull-requests:
        '**':
          - step:
              name: Run PHPUnit tests
              image: pawelgradziel/php73-cli-xdebug-composer:latest
              caches:
                - composer
              script:
                - composer install --no-progress --no-suggest
                - vendor/bin/phpunit --log-junit ./test-reports/phpunit.junit.xml --coverage-clover ./test-reports/phpunit.coverage.xml
              artifacts: # defining the artifacts to be passed to each future step.
                - test-reports/phpunit**
          - step: &step-sonarcloud-scan
              name: Analyze code on SonarCloud
              script:
                - pipe: sonarsource/sonarcloud-scan:0.1.5
                  variables:
                    SONAR_TOKEN: ${SONAR_TOKEN}
                    EXTRA_ARGS: '-Dproject.settings=sonar-project.properties'

Thanks!

I think she said it’s the .sonarcloud.properties file that is currently useless on Bitbucket.
The sonar-project.properties file can still be useful if you have a lot of configuration inside, otherwise if you just have a few you can pass them directly in the EXTRA_ARGS variable of your pipeline like this for exemple: EXTRA_ARGS: '-Dsonar.sources=src -Dsonar.tests=src'

One thing you need to be careful about are the paths in the clover.xml. I had the same issue when I send clover generated by PHPUnit (on WordPress project, but that shouldn’t matter).
The clover.xml that is generated will create file name attribute that is absolute on your system - since I generated my tests locally, the name attribute was something like

/Users/denis/vagrant-local/www/project/public_html/wp-content/plugins/...

So I usually do a search-replace on my clover file so that only wp-content/plugins/... part is left (I delete the first part). Once I send this to SQ in my Jenkins pipeline, the coverage will be shown - not the same as in PHPUnit report, but it will be shown (my guess is that SQ can then generate the coverage internally somehow).

Hope this helps in some way :slight_smile:

Hi Denis,

Thanks for your answer but already got the solution long time ago.

As Grégoire said it was about mocking my tests because was getting a 500 status code.

1 Like