Sonarqube does not detect New Code correctly when using git merge workflow

[Stefan-Alexandru_Pre (Stefan-Alexandru Preda)]

• versions used
  Sonarqube: 8.5 (build 37579)
  Gradle sonarqube plugin: 2.7.1
• error observed (wrap logs/code around triple quote ``` for proper formatting)
  Scanner uses git merge-base HEAD mainline to detect the fork point, all commits starting from that point up to HEAD are considered New Code. However, this doesn’t work in a merge-workflow.
• steps to reproduce

1. Create a branch from mainline: git checkout -b new_branch
2. Create 2 commits, each adding a piece of code.
3. Update the local mainline branch: git checkout master; git pull
   (NOTE): Make sure you pick up at least 1 new commit from origin
4. Merge mainline into the new branch: git checkout new_branch: git merge master
   (NOTE): No merge conflicts expected, no changes to the project we are analyzing are expected either.
5. Create one more commit.
6. Run the gradle sonarqube task. Merge base is detected as the HEAD of mainline, so only the last commit on the branch is considered New Code.

E.g:

$ git --no-pager log --decorate=short --pretty=oneline -n15
113278cd16aa02ca26d54806c46ea4032116ecf7 (HEAD -> DP-6233_potential_fix, origin/DP-6233_potential_fix) DP-6233 - add a git log
bbe1e1f4e190186044dfdb647ca5cfd795b13537 DP-6233 - debug
ce3017c6d0c9b46a5fb571175032c0aeb138eaf2 DP-6233 - add a bit more code
a28d919428001033c8fd2731fbeb40e0eb5246c7 Merge branch 'master' into DP-6233_potential_fix
3250546a4e0bfeefbc96ead9c7186e74ce5c7402 Merge pull request #60507 in PROJECT/repo from IPD-168976-clear-about-me to master
841c801329883a2d4687075ee6ed64eeb2c6fd7d Merge pull request #60368 in PROJECT/repo from DP-6230/qq to master
0422ab9e68b533d474af4577840a1ed399d8696e Merge pull request #60499 in PROJECT/repo from CORE-3879 to master
71dd15129527b03677d38a9028e28f06e2efdb7a Merge pull request #60523 in PROJECT/repo from IPD-169616_Retry_getExperimentVariation to master
df7fe55f6a839d37e7e4b87cc587560f69997726 DP-6233 - add more code
7a4bec492ae4be3c0dbee0fbf6ec4bc2516bfa03 Merge pull request #60387 in PROJECT/repo from IPD-168610 to master
e317ccbe0fab8f2fd4ca5b8b33cead583d62faec DP-6233 - use gradle properties instead of jvm properties
41b7cabd4f73c9e11adfe3753dd617cbbbecd0a2 DP-6233 - update CheckQualityGateTask
c012afc543a2b6ed0504d8f6fc1fa3c3106a71cc DP-6233 - set the sonarqube branch in change promotion
43c7737b8a5f45821428c5ba2cb77ae5c5c8da81 Merge pull request #60185 in PROJECT/repo from IPD-169108 to master
43cd85ff32cb19e831dabf7f00b8f655777f73ce DP-6233 - add some code

All commits starting with DP-6233 are made on the same branch.
Merge base is detected as: 3250546a4e0bfeefbc96ead9c7186e74ce5c7402

2021-01-11T12:49:09.280+0200 [DEBUG] [org.sonarqube.gradle.SonarQubeTask] Merge base sha1: 3250546a4e0bfeefbc96ead9c7186e74ce5c7402

Only commits starting from there, up to HEAD are picked up as New Code

• potential workaround
  Using a rebase-workflow

[Wouter_Admiraal]

Reviving this old thread: @dmeneses do you have any ideas?

[dmeneses (Duarte Meneses)]

Sorry for only having a look at it months later.

I agree that the merge base commit would be the updated mainline, and I think that’s correct following the definition of a merge base.
However, I don’t think that would imply that only the changes in the last commit would be detected.

Simplest reproducer I could create:

git init

echo f1 > f1
git add .
git commit -m f1

git checkout -b HEAD~

echo f2 > f2
git add .
git commit -m f2

git branch feature HEAD~
git checkout feature

echo f3 > f3
git add .
git commit -m f3

git merge master --no-edit

echo f4 > f4
git add .
git commit -m f4

This would result in:

$ git log --decorate --graph --oneline

* cd2753f (HEAD -> feature) f4
*   5b91e49 Merge branch 'master' into feature
|\
| * aa11158 (master) f2
* | 34eab74 f3
|/
* 85c4be5 f1

The merge base is aa11158. The diff between aa11158 and cd2753f isn’t just f4:

$ git diff aa11158 cd2753f --name-only
f3
f4

Could you please post a reproducer that shows the wrong diff being displayed?

[Stefan-Alexandru_Pre (Stefan-Alexandru Preda)]

Hey, thanks for the response!

So I tested the workflow you posted and I got the same results. However, this is what I’m doing that doesn’t look right:

1. Locally in a gradle managed repository, I create a feature branch:

$ git checkout -b feature
Switched to a new branch 'feature'

2. I added this method in a file, in a project

   private void method1() {
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
        System.out.println("This is method1");
    }

3. I commit it

git add .
git commit -m "First commit"

This is the commit tree at this point:

3118e935dcae8605c81d3761f7eb2b9652d00e13 (HEAD -> feature) First commit
82459e42eb58f4e275ad14519b11131174e2d092 (origin/master, origin/HEAD, master) Merge pull request #64986 ...
bebabd3320f95aca7635b37647dfeea0dc0c2f6b Merge pull request #65206 ...
583c2e05ada1f16b3179c2d8ff77f4016cddb733 Merge pull request #65205 ...
8d05c25cac1faaa1e25a4fef243ba8a94c869380 Merge pull request #65173 ...
da5f065f1ff16c0e97dbba2cd50aee8932fa81f3 Merge pull request #65202 ...
0dafcecbbb7999a3cd2e201ea4212abbd70769a3 Merge pull request #65188 ...
7493accb1cd77c88f4109460e11aa6a0212cb486 Merge pull request #65155 ...
ed3e4b302327681fbe95584b2ebad20c0d034ed4 Merge pull request #65187 ...
686ca05cbd8141337855b93d249fb7b8ba5f4fca Merge pull request #65149 ...
b6e266d19574c5190e15d81f3cc0e1f2b6d2bf13 Merge pull request #65185 ...
743cb7a0d06537bc0af1b477feb3ffdcfc52fa7e Merge pull request #65130 ...
4f52d5e3f2f05a481fee33d256a3e88eac29861d Merge pull request #65186 ...
9f8fe6a0e1b31c5c55815ed2b284a4282095d048 Merge pull request #65162 ...
3b9bfc19d0eadd8cd07af9dbedbe50a744badc5a Merge pull request #64980 ...

4. I run the sonarqube gradle task

gradlew :project:sonarqube --debug
...
2021-06-07T13:37:17.263+0300 [DEBUG] [org.sonarqube.gradle.SonarQubeTask] Merge base sha1: 82459e42eb58f4e275ad14519b11131174e2d092

Looks right.

5. Now checking results in Sonarqube:
   
   

Perfect so far.

6. Going back to the git repo, I switch to master, pull from origin, switch to feature and merge with master

$ git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
$ git pull
remote: Enumerating objects: 328, done.
remote: Counting objects: 100% (328/328), done.
remote: Compressing objects: 100% (210/210), done.
remote: Total 328 (delta 89), reused 114 (delta 42)
Receiving objects: 100% (328/328), 93.04 KiB | 727.00 KiB/s, done.
Resolving deltas: 100% (89/89), completed with 22 local objects.
...
$ git checkout feature
Switched to branch 'feature'
$ git merge master
...

At this point, this is how the commit tree looks like:

$ git --no-pager log --decorate=short --pretty=oneline -n15
05e89f9599fe352054716237e849f4467c8420df (HEAD -> feature) Merge branch 'master' into feature
d14190f6a89222fcd637e11cc4eb6745020344f5 (origin/master, origin/HEAD, master) Merge pull request #65124 ...
3118e935dcae8605c81d3761f7eb2b9652d00e13 First commit
e28fa35453595c8654f6872b7a195bc7e80b4c2e Merge pull request #65170 ...
5090eba07af834e10a1bdd550a4cea72d0d73108 Merge pull request #65182 ...
82459e42eb58f4e275ad14519b11131174e2d092 Merge pull request #64986 ...
bebabd3320f95aca7635b37647dfeea0dc0c2f6b Merge pull request #65206 ...
583c2e05ada1f16b3179c2d8ff77f4016cddb733 Merge pull request #65205 ...
8d05c25cac1faaa1e25a4fef243ba8a94c869380 Merge pull request #65173 ...
da5f065f1ff16c0e97dbba2cd50aee8932fa81f3 Merge pull request #65202 ...
0dafcecbbb7999a3cd2e201ea4212abbd70769a3 Merge pull request #65188 ...
7493accb1cd77c88f4109460e11aa6a0212cb486 Merge pull request #65155 ...
ed3e4b302327681fbe95584b2ebad20c0d034ed4 Merge pull request #65187 ...
686ca05cbd8141337855b93d249fb7b8ba5f4fca Merge pull request #65149 ...
b6e266d19574c5190e15d81f3cc0e1f2b6d2bf13 Merge pull request #65185 ...

7. Now I add some more code and commit it.
   This is what I add:

    private void method2() {
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
        System.out.println("This is method2");
    }

And I commit it:

$ git add .
$ git commit -m "Second commit"
[feature d7c2caef89c] Second commit
 1 file changed, 27 insertions(+)

This is how the commit tree looks like:

$ git --no-pager log --decorate=short --pretty=oneline -n15
d7c2caef89c4171a5f58ea284ec5b80998f27336 (HEAD -> feature) Second commit
05e89f9599fe352054716237e849f4467c8420df Merge branch 'master' into feature
d14190f6a89222fcd637e11cc4eb6745020344f5 (origin/master, origin/HEAD, master) Merge pull request #65124 in SITE/weightsite from IPD-179180 to master
3118e935dcae8605c81d3761f7eb2b9652d00e13 First commit
e28fa35453595c8654f6872b7a195bc7e80b4c2e Merge pull request #65170 ...
5090eba07af834e10a1bdd550a4cea72d0d73108 Merge pull request #65182 ...
82459e42eb58f4e275ad14519b11131174e2d092 Merge pull request #64986 ...
bebabd3320f95aca7635b37647dfeea0dc0c2f6b Merge pull request #65206 ...
583c2e05ada1f16b3179c2d8ff77f4016cddb733 Merge pull request #65205 ...
8d05c25cac1faaa1e25a4fef243ba8a94c869380 Merge pull request #65173 ...
da5f065f1ff16c0e97dbba2cd50aee8932fa81f3 Merge pull request #65202 ...
0dafcecbbb7999a3cd2e201ea4212abbd70769a3 Merge pull request #65188 ...
7493accb1cd77c88f4109460e11aa6a0212cb486 Merge pull request #65155 ...
ed3e4b302327681fbe95584b2ebad20c0d034ed4 Merge pull request #65187 ...
686ca05cbd8141337855b93d249fb7b8ba5f4fca Merge pull request #65149 ...

8. Running the sonarqube task again:

gradlew :project:sonarqube --debug
...
2021-06-07T13:43:27.273+0300 [DEBUG] [org.sonarqube.gradle.SonarQubeTask] Merge base sha1: d14190f6a89222fcd637e11cc4eb6745020344f5

Looks right.

9. Now checking results in Sonarqube:
   
   

Well, looks like it only detected the changes from my 2nd commit as New code. You can actually see a chunk of the 1st method I created at the top of the screenshot.

Also these are the project settings for New Code:

We have a gradle plugin which figures out the branch automatically and sets it like this:

project.sonarqube {
                properties {
                    ...
                    property "sonar.branch.name", "${project.gitRevisions.branch}"

Anyway, it looks like it figured it out correctly, judging by what I see in Sonarqube and the debug logs:

...
ANALYSIS SUCCESSFUL, you can browse https://sonarqube.site-ops.fitbit.com/dashboard?id=project&branch=feature
...

This is the full list of sonarqube settings in the gradle plugin:

project.sonarqube {
                properties {
                    property "sonar.projectKey", "${project.ids.sonarKey}"
                    property "sonar.projectName", "${projectName}"
                    property "sonar.branch.name", "${project.gitRevisions.branch}"
                    property "sonar.sourceEncoding", "UTF-8"
                    property "sonar.test.inclusions", "**/*Test*/**"
                    property "sonar.language", "java"
                    property "sonar.coverage.jacoco.xmlReportPaths", [project.fileTree(dir: "${project.buildDir}/reports/jacoco", include: '**/*.xml')]
                    property "sonar.java.binaries", "$project.buildDir/classes"
                    property "sonar.host.url", "${project.property("systemProp.sonar.host.url")}"
                    property "sonar.dependencyCheck.reportPath", dependencyReportPath
                    property "sonar.dependencyCheck.htmlReportPath", "${project.buildDir.path}/reports/dependency-check-report.html"
                    // extend exclusions set by projects
                    properties["sonar.exclusions"] = (properties["sonar.exclusions"] ?: []) + ["**/*Test*/**","**/generatedsrc/**","**/generated-src/**", "**/gen-src/**", "**/generated/main/**", "**/generated/test/**"]
                }
            }

Best regards,
Stefan

[dmeneses (Duarte Meneses)]

Thanks! I thought it was a pull request analysis, not a branch with the New Code set to “reference branch”.

I was able to reproduce and I understand now the cause of the problem.

To compute the lines changed in branches using a “reference branch”, the scanner doesn’t actually do a git diff. Instead, it finds the merge base and assumes that any line modified after the merge-base is new (by comparing commit dates).
In your case, the merge-base is more recent than the first commit in feature, so the changes in that first commit are not included in the New Code.

The analysis of pull requests does a git diff so doesn’t have this problem.

I opened a bug ticket with more details: https://jira.sonarsource.com/browse/SONAR-14929