Thanks for confirming the issue is related to core.autocrlf on Windows. To avoid this issue, I think we should use (or emulate) the -w option of git diff to ignore any whitespace change when considering line changes (which are most of the time not taking any role in issues).
It would also be more consistent with the blame.
I created a ticket:
https://jira.sonarsource.com/browse/SONARSCGIT-41
I set the respective config option in the .gitattributes file in the repo itself. Maybe the Git plugin does not read this file or the settings on the host have priority?
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto
It is maybe a limitation of JGit, but we will dig more into it when tackling the ticket I created. In the meantime you’ll have to configure your CI boxes to not alter line ends when checking out the code.
@Julien_HENRY @Andreas_Petersen I have the same issue on Windows and the fix ended up being setting git config --global core.autocrlf true for me.
@Julien_HENRY This ticket may be related as jGit should be reading properties from .gitattributes file on the repo.
I don’t think this ticket is related (it talk about the archive command that works remotely). But still I found several other tickets talking about limited support for some attributes in .gitattributes. Again we will dig into it when talking the ticket I created.
I’ve been trying a few different things, and as mentioned in my previous comment I was able to make it work when setting core.autocrlf=false. In this case I made a new commit with the setting set to false, and then ran the SonarQube scan.
I have also had success with the setting set to true, odly enough. I haven’t investigated in detail why this is, since I do have a working setup now. What I think is worth pointing out is, that even when I was experiencing the problem originally, I did have core.autocrlf=true in C:/ProgramData/Git/config, as you can see in my previous posts. When I set git config --global core.autocrlf true, it adds the setting to %HOMEDRIVE%/.gitconfig. So it would seem that the Git plugin doesn’t get the setting from C:/ProgramData/Git/config but does get the setting from %HOMEDRIVE%/.gitconfig.
Thanks a lot for the hint i got it finally working too:
git config --get core.autocrlf printed truegit config --global core.autocrlf false as suggested (goes into ~/.gitconfig) → diff still for the whole filegit config --global core.autocrlf true → correct diffsSo i can confirm that SonarQube/JGit reads the ~/.gitconfig only and it seems that it has a different default config value for core.autocrlf than the native Git.
Thanks for the feedback. To me it kind of make sense JGit doesn’t consider C:/ProgramData/Git/config since it is an “implementation specific location”. Out of curiosity, how did you ended up having a setting there? Is it a feature specific to the Windows Git client?
I installed Git for Windows via the installer, and there is a window where it asks you what you want for the setting. So I imagine the Git for Windows installer sets the setting in that directory.
When I try and run the command git config --global core.autocrlf true on our Windows box it just complains that git is not a recognised command.
'git' is not recognized as an internal or external command,
operable program or batch file.Do I need to install git for Windows on this box just to be able to set this property?
We use Azure Devops with our own agents. When running the analysis in verbose mode it tells me it might be looking in the agent folder at the gitattributes file, however running the above command in these folders has made no difference…
1:32:08.245 DEBUG: readpipe [git, config, --system, --edit],D:\Mainframe\externals\git\cmd
11:32:08.463 DEBUG: readpipe may return 'D:/Mainframe/externals/git/mingw64/etc/gitconfig'Does anyone have any ideas?
Thanks,
Pete
@pconnor I think you should be able to create the %HOMEDRIVE%/.gitconfig and write to the file manually:
[core]
autocrlf = true
Including any other configuration you want.
@Andreas_Petersen No, unfortunately that didn’t work. When I look at the agents git config file (I was looking at gitattributes before). I can see the following
[core]
symlinks = false
autocrlf = trueSo that suggests it is in there… Maybe I have a slightly different issue to others in that I’m using self hosted build agents for Azure Devops. @Julien_HENRY do you have any ideas on where this setting could be that I need to update?
Thanks,
Pete
Checking Github repository for SCM plugin, looks fixed the issue related to the whitespaces
But the plugin update is not yet available on the marketplace (still 1.8 build 1574). Someone know why?
In my case the workaroud
git config --global core.autocrlf false
worked for a while and actually no more (tried to set at project level and also globally)
Hi Luigi,
We’ll publish it in the update center tomorrow.
Meanwhile, you can download it here: https://repox.jfrog.io/repox/sonarsource/org/sonarsource/scm/git/sonar-scm-git-plugin/1.9.0.1725/sonar-scm-git-plugin-1.9.0.1725.jar
Just drop it into extension/plugins and restart the server.
Thanks! I have updated the extension plugin as suggested and now is the version 1.9.0.1725, but I still have the issue reported by all the users. Nothing is changed with this update.
Can you suggest me how to fix it?
To confirm the problem is related to autocrlf, does it work if you set autocrlf to true, either at project level or global (for example git config --global core.autocrlf true)?
I tested globally and locally the git command before to execute the build on the self-hosted Azure DevOps agent (that is not on the same server machine of SonarQube server)
git config --global core.autocrlf true
git config core.autocrlf true
git config --get core.autocrlf
and set to true/false does not change file difference reported.
With the previous version (1.8.0) were reported also other differences (on other files) setting core.autocrlf to false.
2019-07-16T18:47:31.3732805Z
2019-07-16T18:47:31.3733147Z _ Git Config Global CHANGED
2019-07-16T18:47:31.3970355Z user.name=xxxxx
2019-07-16T18:47:31.3970975Z user.email=xxxx
2019-07-16T18:47:31.3971137Z credential.helper=manager
2019-07-16T18:47:31.3971217Z credential.interactive=never
2019-07-16T18:47:31.3971270Z core.autocrlf=true
2019-07-16T18:47:31.3971323Z core.repositoryformatversion=0
2019-07-16T18:47:31.3971388Z core.filemode=false
2019-07-16T18:47:31.3971438Z core.bare=false
2019-07-16T18:47:31.3971484Z core.logallrefupdates=true
2019-07-16T18:47:31.3971546Z core.symlinks=false
2019-07-16T18:47:31.3971627Z core.ignorecase=true
2019-07-16T18:47:31.3971673Z core.fscache=true
2019-07-16T18:47:31.3971739Z color.diff=auto
2019-07-16T18:47:31.3971782Z color.status=auto
2019-07-16T18:47:31.3971828Z color.branch=auto
2019-07-16T18:47:31.3971894Z color.interactive=true
2019-07-16T18:47:31.3971940Z help.format=html
2019-07-16T18:47:31.3972002Z rebase.autosquash=true
2019-07-16T18:47:31.4862783Z Git Config Locally CHANGED
2019-07-16T18:47:31.5103504Z core.symlinks=false
2019-07-16T18:47:31.5104019Z core.autocrlf=true
2019-07-16T18:47:31.5104096Z color.diff=auto
2019-07-16T18:47:31.5104199Z color.status=auto
2019-07-16T18:47:31.5104416Z color.branch=auto
2019-07-16T18:47:31.5105304Z color.interactive=true
2019-07-16T18:47:31.5105592Z pack.packsizelimit=2g
2019-07-16T18:47:31.5105682Z help.format=html
2019-07-16T18:47:31.5106049Z http.sslcainfo=xxxxx
2019-07-16T18:47:31.5106108Z diff.astextplain.textconv=astextplain
2019-07-16T18:47:31.5106182Z rebase.autosquash=true
2019-07-16T18:47:31.5106232Z user.name=xxxxx
2019-07-16T18:47:31.5106278Z user.email=xxxx
2019-07-16T18:47:31.5106353Z credential.helper=manager
2019-07-16T18:47:31.5106402Z credential.interactive=never
2019-07-16T18:47:31.5106464Z core.autocrlf=true
2019-07-16T18:47:31.5106511Z core.repositoryformatversion=0
2019-07-16T18:47:31.5106557Z core.filemode=false
2019-07-16T18:47:31.5106623Z core.bare=false
2019-07-16T18:47:31.5106688Z core.logallrefupdates=true
2019-07-16T18:47:31.5106734Z core.symlinks=false
2019-07-16T18:47:31.5106800Z core.ignorecase=true
2019-07-16T18:47:31.5106849Z core.fscache=true
2019-07-16T18:47:31.5106896Z color.diff=auto
2019-07-16T18:47:31.5106952Z color.status=auto
2019-07-16T18:47:31.5106995Z color.branch=auto
2019-07-16T18:47:31.5107042Z color.interactive=true
2019-07-16T18:47:31.5107113Z help.format=html
2019-07-16T18:47:31.5107156Z rebase.autosquash=true
2019-07-16T18:47:31.5107203Z core.repositoryformatversion=0
2019-07-16T18:47:31.5107265Z core.filemode=false
2019-07-16T18:47:31.5107327Z core.bare=false
2019-07-16T18:47:31.5107389Z core.symlinks=false
2019-07-16T18:47:31.5107436Z core.ignorecase=true
2019-07-16T18:47:31.5107482Z core.autocrlf=true
2019-07-16T18:47:31.5107557Z remote.origin.url=xxxxxx
2019-07-16T18:47:31.5107619Z remote.origin.fetch=+refs/heads/:refs/remotes/origin/
2019-07-16T18:47:31.5108373Z gc.auto=0
2019-07-16T18:47:31.5132319Z Git config value core.autocrlf
2019-07-16T18:47:31.5340305Z true
Ok, that indicates that the problem is probably something else.
Could you please post the logs of the scanner, with debug enabled?
Attached the SonarScanner logs (prepare + scanner).-removed-