SonarQube detects whole file as new code in shortlived branches and pull requests, when only part of the file has changed

SonarQube
1

Hi,

We’re evaluating SonarQube 7.7. developer edition in my team. We’re scanning Java code, and we have installed the PMD and Findbugs plugins. Here are some version information:

  • SonarQube, 7.7.0.23042
  • SonarJava, 5.11 (build 17289)
  • Findbugs, 3.10.0
  • PMD, 3.2.0
  • Git, 1.8 (build 1574)
  • Gradle sonarqube scanner, 2.7

We are having issues with branch analysis, for short lived branched and pull requests. Specifically, we see that if we change even just 1 line in a file, the whole file is detected as changed. This means that if we’re adding a new method to a class, that we have fully tested, if the remaining unchanged file was not tested, then we do not reach the coverage target.

I tested this by:

  • Making a new branch

  • Add a new line a method that is covered by tests, such that the new line is also covered by tests. The remaining methods in the class do not have test coverage. Thus, coverage for the whole file is less than 80 %, but coverage on the new code should be 100 %.

  • Commit the changes

  • Run gradle sonarqube with the following properties:

      sonarqube {
      properties {
          property 'sonar.projectKey', "${project.name}"
          property 'sonar.host.url', 'SERVER_LOCATION'
          property 'sonar.login', 'TOKEN'
          property 'sonar.branch.name', 'sonarqube-test'
          property 'sonar.branch.target', 'master'
          property 'sonar.scm.provider', 'git'
      }
  }

Observations:

  • If I specify the branch as a shortlived branch or pull request, then the whole file will be considered new code, despite the fact that I only changed 1 line in the file. The implications are that the analysis will fail, since the unchanged code that is incorrectly detected as new code does not have sufficient code coverage.
  • If I specify the branch as a longlived branch, then the new code is correctly detected as only the 1 new line I added. However, then we cannot use the features of shortlived branches and pull requests. We’re mostly interested in the shortlived branches, since that is how we work, and addtionally the pull requests.

Here’s a comparison image (I had to make it one image, since new users can only upload one image):

image
image.png1020×1023 101 KB

In the top analysis I changed one line that has 100 % code coverage, which resulted in the whole file being detected as changed. This results in less than 80 % code coverage, failing the quality gate. For the bottom analysis I changed two lines that have 100 % code coverage. As you can see, SonarQube correctly detects that two lines were changes and that they have a coverage of 100 %, meaning that the quaility gate passes.

Is this the intended behaviour? We’re quite confused in our team, as we would expect only the changed line to be considered new code, not the whole file. It prevents us from introducing pull request integration and using shortlived branches, since it’s not viable to add tests for lines we did not actually change.

1 Like
2

For short lived branches I have done much better using the pull request properties instead of the branch properties:
https://docs.sonarqube.org/latest/analysis/pull-request/

3

Hi Ryan,

I’ve tried using the pull request properties as well, but that gives the same effect: Changing just 1 line in a file, marks the whole file as new code.

4

Hi Guys,

This seems to be the same problem that’s being investigated in

If you agree, in order to keep the investigation focused, could you follow along/contribute there, please?

 
Thx,
Ann

2 Likes
5

We are having same situation with Azure Devops SonarSource, but not for every file in the PR. Some of then are correctly marked, and others are reflected as new, although they only have 1 line change.

6

Is there any update on this bug? I’m having the same issue

7

There is a chance that it happens because of line endings are different between commits

e.g. imagine that you got your file with \n, change single line, and commit it back but now with \r\n

diff tools will show you only this one line change, but actually absolutelly every line is changed

this might be a reason for sonar to start checking whole file instead of single line

1 Like