Whole files considered new code in short-lived branch

I can try to make a dummy repo that reproduces the issue that I could share

That would be awesome!

Hi @Julien_HENRY,

I have reproduced the issue in a new repo, you can find it here: https://github.com/AndreasPetersen/sonarqube-change. The repo has the steps to reproduce.

Additionally, here are some further screenshots of my results (now that I have some code I can actually show):

In the screenshot below are the results after the initial scan on master:

Below are the results of the scan on a branch, where a single line was changed:

And the file itself:

In the git diff I have:

git show
commit 81e87bc2d7827c588b8aed4954ae055a2ed4f489 (HEAD -> sonarqube-test)
Author: Andreas Petersen 
Date:   Mon Apr 15 13:32:24 2019 +0200

    made a change

diff --git a/src/main/java/dk/andreas/Main.java b/src/main/java/dk/andreas/Main.java
index f97860f..afaae6b 100644
--- a/src/main/java/dk/andreas/Main.java
+++ b/src/main/java/dk/andreas/Main.java
@@ -15,6 +15,7 @@ public class Main {
    }

    public static boolean tested() {
+        System.out.println("Adding a line");
        return true;
    }
}

As you can see I only changed one line in the file, yet SonarQube detects the whole file as changed.

2 Likes
  • git checkout -b sonarqube-test
  • Add a line in Main.tested() , e.g. a print statement just before the return statement
  • gradle build

Don’t you commit/push the change in the branch? We are supporting analysis of code in branch/PR that have been pushed to the ALM. Analyzing local changes may lead to unexpected results, since we can’t use integration with the ALM to request missing info.

Ah yes, sorry, I forgot writing the commit step

I updated the README to include the commit step.

Just to be clear regarding my previous comment: When I wrote “I forgot writing the commit step”, I mean I forgot writing it in the README on the repo. I did commit, as you can see from my git show in my other previous comment.

Works fine for me:

Is the Git plugin up to date in your SonarQube instance? Mine is 1.8 (build 1574)

What is your operating system? Do you have some global Git settings that could edit all lines (like auto crlf)?

The operating system that SonarQube runs on is: Windows 7 Enterprise, Version 6.1 (Build 7601: Service Pack 1)
The operating system that I run the scanner on (my computer): Windows 7 Enterprise, Version 6.1 (Build 7601: Service Pack 1)
Version of the Git plugin: 1.8 (build 1574)

These are my git settings:

file:"C:\\ProgramData/Git/config"       core.symlinks=false
file:"C:\\ProgramData/Git/config"       core.autocrlf=true
file:"C:\\ProgramData/Git/config"       core.fscache=true
file:"C:\\ProgramData/Git/config"       color.diff=auto
file:"C:\\ProgramData/Git/config"       color.status=auto
file:"C:\\ProgramData/Git/config"       color.branch=auto
file:"C:\\ProgramData/Git/config"       color.interactive=true
file:"C:\\ProgramData/Git/config"       help.format=html
file:"C:\\ProgramData/Git/config"       rebase.autosquash=true

and the git settings in the repo itself:

file:.git/config        core.repositoryformatversion=0
file:.git/config        core.filemode=false
file:.git/config        core.bare=false
file:.git/config        core.logallrefupdates=true
file:.git/config        core.symlinks=false
file:.git/config        core.ignorecase=true

Setting git config --global core.autocrlf false fixed the issue. It now works. Thanks @Julien_HENRY. Perhaps the documentation on branches needs an update to include this information, or perhaps I just missed it?

Thanks for confirming the issue is related to core.autocrlf on Windows. To avoid this issue, I think we should use (or emulate) the -w option of git diff to ignore any whitespace change when considering line changes (which are most of the time not taking any role in issues).
It would also be more consistent with the blame.

I created a ticket:
https://jira.sonarsource.com/browse/SONARSCGIT-41

I set the respective config option in the .gitattributes file in the repo itself. Maybe the Git plugin does not read this file or the settings on the host have priority?

# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto

It is maybe a limitation of JGit, but we will dig more into it when tackling the ticket I created. In the meantime you’ll have to configure your CI boxes to not alter line ends when checking out the code.

@Julien_HENRY @Andreas_Petersen I have the same issue on Windows and the fix ended up being setting git config --global core.autocrlf true for me.

@Julien_HENRY This ticket may be related as jGit should be reading properties from .gitattributes file on the repo.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=545701

I don’t think this ticket is related (it talk about the archive command that works remotely). But still I found several other tickets talking about limited support for some attributes in .gitattributes. Again we will dig into it when talking the ticket I created.

I’ve been trying a few different things, and as mentioned in my previous comment I was able to make it work when setting core.autocrlf=false. In this case I made a new commit with the setting set to false, and then ran the SonarQube scan.

I have also had success with the setting set to true, odly enough. I haven’t investigated in detail why this is, since I do have a working setup now. What I think is worth pointing out is, that even when I was experiencing the problem originally, I did have core.autocrlf=true in C:/ProgramData/Git/config, as you can see in my previous posts. When I set git config --global core.autocrlf true, it adds the setting to %HOMEDRIVE%/.gitconfig. So it would seem that the Git plugin doesn’t get the setting from C:/ProgramData/Git/config but does get the setting from %HOMEDRIVE%/.gitconfig.

Thanks a lot for the hint i got it finally working too:

  1. Before changing anything git config --get core.autocrlf printed true
  2. I tried git config --global core.autocrlf false as suggested (goes into ~/.gitconfig) -> diff still for the whole file
  3. After git config --global core.autocrlf true -> correct diffs

So i can confirm that SonarQube/JGit reads the ~/.gitconfig only and it seems that it has a different default config value for core.autocrlf than the native Git.

2 Likes

Thanks for the feedback. To me it kind of make sense JGit doesn’t consider C:/ProgramData/Git/config since it is an “implementation specific location”. Out of curiosity, how did you ended up having a setting there? Is it a feature specific to the Windows Git client?

I installed Git for Windows via the installer, and there is a window where it asks you what you want for the setting. So I imagine the Git for Windows installer sets the setting in that directory.

When I try and run the command git config --global core.autocrlf true on our Windows box it just complains that git is not a recognised command.

'git' is not recognized as an internal or external command,
operable program or batch file.

Do I need to install git for Windows on this box just to be able to set this property?

We use Azure Devops with our own agents. When running the analysis in verbose mode it tells me it might be looking in the agent folder at the gitattributes file, however running the above command in these folders has made no difference…

1:32:08.245 DEBUG: readpipe [git, config, --system, --edit],D:\Mainframe\externals\git\cmd
11:32:08.463 DEBUG: readpipe may return 'D:/Mainframe/externals/git/mingw64/etc/gitconfig'

Does anyone have any ideas?

Thanks,
Pete

1 Like