Analysis is running on the whole code base not only the changes in the PR

Hello there,
I am writing a Github action to analyze the code GitHub - mautic/mautic: Mautic: Open Source Marketing Automation Software., I managed to make it analyze PRs from forks. I also defined the the variables to make it scan the files changed in the PR itself but it is not picking that up.

Here is the code for github actions

And here is an example of the run

For reference

args: -Dsonar.organization=mautic -Dsonar.projectKey=mautic_mautic -Dsonar.links.homepage=https://github.com/mautic/mautic -Dsonar.links.ci=https://github.com/mautic/mautic/actions/workflows/tests.yml -Dsonar.links.scm=https://github.com/mautic/mautic -Dsonar.links.issue=https://github.com/mautic/mautic/issues -Dsonar.sourceEncoding=UTF-8 -Dsonar.sources=app,plugins -Dsonar.projectBaseDir=. -Dsonar.inclusions=app/*.php,app/**/*.php,plugins/*.php,plugins/**/*.php -Dsonar.exclusions=app/migrations/**/*,app/bundles/*Bundle/Config/**/*,app/bundles/*Bundle/DataFixtures/**/*,app/bundles/*Bundle/Tests/**/*,app/bundles/*Bundle/Translations/**/*,app/bundles/*Bundle/Views/**/*,app/middlewares/Test/**/*,app/bundles/CoreBundle/Test/**/*,plugins/*Bundle/Config/**/*,plugins/*Bundle/Tests/**/*,plugins/*Bundle/Translations/**/*,plugins/*Bundle/Views/**/* -Dsonar.cpd.exclusions=**/* -Dsonar.tests=tests -Dsonar.php.coverage.reportPaths=coverage/coverage.xml -Dsonar.php.tests.reportPath=coverage/junit.xml -Dsonar.scm.disabled=true -Dsonar.verbose=true -Dsonar.scm.revision=2c160f2af9754f49a1bf3551cc34ef31c5af1d8e -Dsonar.pullrequest.key=11613 -Dsonar.pullrequest.branch=add_mailer -Dsonar.pullrequest.base=5.x -Dsonar.newCode.referenceBranch=5.x

Hi,

Welcome to the community!

It looks like you’re using v1.8 of the action, and v1.9.1 is the latest. An upgrade might help. Or just let it default as in this example?

Regarding the PR parameters, I believe they should be read from the environment and automatically filled in for you.

As a side note, I noticed this in your pipeline: -Dsonar.scm.disabled=true. Particularly if you’re trying to get accurate PR analysis, you really don’t want to disable the SCM data collection. Without SCM data, analysis can’t tell what’s new in your PR and will report ev-ry-thing.

 
HTH,
Ann

Hi,

Thanks for your answer, I made some changes to the action description it now uses master version for the action, I also turned -Dsonar.scm.disabled=false.

Regarding, the PR parameters they are not picked up automatically because the come from external fork

I will merge and let you know the results

Thanks @ganncamp ,

I made the changes you suggested in the same Github workflow linked above, but unfortunately it did not pick up the changed lines, you can see the run log here

I do not know what is still missing?

Hi,

In the most recent log (BTW, DEBUG logging is a bit much) I was eventually able to find this:

2023-06-15T13:14:41.6215381Z 13:14:41.534 DEBUG: SCM reported changed lines for 0 files in the branch
2023-06-15T13:14:41.6215527Z 13:14:41.534 INFO: SCM writing changed lines (done) | time=48ms

What code files are changed in this branch/PR?

 
Ann

Thanks @ganncamp for your time.

Yes, all the PR got analyzed has changed files but the scan does not understand there is a change.

Here are some of the PRs

https://github.com/mautic/mautic/pull/12459
Fixing migrations by escopecz · Pull Request #12491 · mautic/mautic · GitHub

Would be great if we can get sonar working for Mautic, the community is getting bigger and developers would appreciate the analysis

Thank you for your help

Hi,

Thanks for the pointer to the PR. I’m having a hard time finding the logs of its SonarCloud analysis. Could you point me to them? And to the PR on SonarCloud?

 
Thx,
Ann

Hi @ganncamp ,

Okay, to remove the confusion let’s just study this PR in specific

This PR changed 186 files, but the analysis did not show any difference.

Appreciate your input on that

1 Like

Hi,

Thanks for the very concrete example. You have a lot of files in this PR and a lot of exclusions, but the exclusions only eliminate a few of them, and with the logs and the SonarCloud URL the logs provide, I’m able to verify that SonarCloud is indeed not finding any changed files. I can’t see anything obvious here, so I’ve flagged this for more expert eyes.

 
Ann

Hi,

Thank you, I think if you check the other steps in the run would be great, we are using a GitHub actions that gets the details of the PR include last commit sha.

So that might helpful during your debug

Thank you for your support

1 Like

Hi @ganncamp ,

I disabled the workflow for now, if you get any feedback, please let me know we will implement it immediately

Thanks

1 Like

Hi @mabumusa1,

The 2 PRs linked in this thread come from external forks.
Sadly, analyzing external pull requests is not supported yet in SonarCloud.
I would suggest that you vote for this feature candidate to help us build the SonarCloud roadmap:
https://portal.productboard.com/sonarsource/1-sonarcloud/c/50-sonarcloud-analyzes-external-pull-request

I also spotted that in the logs:

13:04:43.974 DEBUG: Exception caught during execution of command '[/usr/bin/git, config, --system, --show-origin, --list, -z]' in '/usr/bin', return code '128', error message 'fatal: unable to read config file '/etc/gitconfig': No such file or directory

The SonarScanner relies on the JGit library to detect the diff. It means the Scanner needs to have access to the Git configuration, and have the target branch history to be able to compute the diff. It may explain why the changed files are not seen.

Hope that helps,
Claire