SonarCloud analysis for different PR's with same base branch are resulting different analysis

SonarCloud
,
1

On our private repository (GitHub) we created a branch release-4.11 from the develop branch. After finished all the development we created 2 PRs:

  1. The first one to merge all changes back to the development branch:
    sonar-scanner -X -D project.settings=sonar-project.properties -D sonar.pullrequest.key=143 -D sonar.pullrequest.branch=release-4.11 -D sonar.pullrequest.base=development -D sonar.login="$SONAR_TOKEN"

image
image923Ă—267 21.1 KB

  1. The second one to merge all changes into the master branch:
    sonar-scanner -X -D project.settings=sonar-project.properties -D sonar.pullrequest.key=150 -D sonar.pullrequest.branch=release-4.11 -D sonar.pullrequest.base=master -D sonar.login="$SONAR_TOKEN"

image
image923Ă—267 21.8 KB

As we can see on the images above the results were different for both executions but the problem is that for the merge into the development branch a lot of files that were not changed was marked as an issue (bug, code smell, et cetera)

image
image742Ă—173 21.5 KB

On the SonarCloud PR analysis, I have 2 warning that could impact this or not, I don’t know:

image

Details:
ALM used: GitHub
CI system used: GitHub Actions
Languages of the repository: Objective-C
Steps to reproduce: Execute pull request sonar-scanner execution to SonarCloud with the same branch and different targets branches.
Potential workaround: ?

2

Hi @vhsantos26

Maybe dumb question but : are you sure that your “develop” branch is called “development” ?

If yes, looks like the git clone action before starting the analysis doesn’t fetch relevant information from your repo, you should check this first.

Mickaël

3

Hi @mickaelcaro

There are no dumb questions hehe and yes, I am sure that our develop branch is development. (Don’t ask me why hahaha)

And about the git clone today is configurated like this:

- name: Checkout
    uses: actions/checkout@v2
    with:
      fetch-depth: 0
      submodules: "true"
      token: ${{ secrets.ACCESS_TOKEN }}

Another thing maybe is causing this problem or not but can help us understand it. We have the development branch and from this one, we created the release-4.11 branch. Now we are merging it (release-4.11) back to the development branch again and into the master branch. Our pipeline is set to run Sonar-Scanner using this command for both executions (PR analysis): -D sonar.pullrequest.branch=${GITHUB_HEAD_REF} -D sonar.pullrequest.base=${GITHUB_BASE_REF}. I imagine that the base branch will be the one that we are merge into.

6

Hi @vhsantos26,

Both of these warnings are very much relevant. (We really try to make our warnings relevant. When you don’t fully understand what they mean, and their implications, it’s important to get a full understanding before proceeding, because otherwise you may see many inexplicable things that don’t make sense.)

The first warning means that the scanner couldn’t find the development branch in the Git working tree where it was executed. Without the development branch, it’s not possible to identify the files and the lines that changed between the development branch and the current working tree. We have a fallback behavior based on timestamps, but that typically won’t match perfectly the actual changes that happened in the Git repo. The consequence is precisely what you are experiencing: there are issues reported on files that haven’t changed, as per Git. The scanner couldn’t know that, because it couldn’t get the development ref.

The second warning is closely related. The scanner detected a shallow clone. In a shallow clone, git blame annotation information is typically incomplete, and also if the shallow clone doesn’t have enough depth to include the common ancestor commit of the current working tree and the development branch, then again it won’t be possible to identify the changed files and lines, resulting in the same effect as I explained for the other warning.

The typical solution for both of these warnings when using GitHub Actions is to activate fetch-depth: 0, as in your last post. With this parameter in place, the scenario you described, and the warnings, should not be possible. The fetch-depth: 0 parameter ensures the clone is not shallow, and it fetches all branches, therefore development should be found at origin/development.

Are you sure that fetch-depth: 0 is in the action description of the analyses where you get these warnings? If yes, then it seems it’s not effective somehow. If it was, you wouldn’t have the issue. Keep in mind that the action description is in a file that’s part of the repository, so you may have different content in different branches. I suggest to check the log output of the action, I think it should be visible if it’s really a shallow clone or not. To verify that the development branch was fetched, you could temporarily add a git branch -r command. If you can fix your build scripts so that origin/development shows up in the output, then the scanner should be able to find what it needs, the warnings should go away, and you should see expected results on SonarCloud.

1 Like
7

Hi @janos,

First things first, thank you for your explanation, helped me a lot to understand how the warnings and how those are important.

Now I am because before I’ve included the fetch-deoth: 0 on the first Job checkout on GitHub Actions and this one was not the right one. Now I’ve included this on the job where the SonarCloud runs the following log:

2020-12-15T17:53:51.8881320Z * [new branch] development -> origin/development

With that, we don’t have those warnings anymore (for both the master and development branches) anymore and the results were quite different.

Master branch: Now we have the right report and it is showing issues where we really changed the code.

Development branch: The bugs and code coverage now are looks okay but we still seeing code smells (70 code smells) where we did not change (we changed the file but not the line where the code smell is)

image
image827Ă—144 21.1 KB

I am uploading the Sonar-Scanner execution for the Analysis ID AXZnlwrSDnpcfiP5dzzB

12_Sonar Pull Request Analysis.txt.zip (546.0 KB)

8

Judging by the name of the branch, it seems to me that this is in fact not a pull request analysis, but a long-branch analysis. In which case it’s normal to see issues on old code.

Let me know if you still think some issues are incorrectly reported on old code, and we can reboot this investigation.