Pull request decorating failure

Must-share information (formatted with Markdown):

  • which versions are you using
    (SonarQube - Version 8.9 (build 43852),
    Scanner - sonar-scanner-3.0.3.778,
    TeamCity Professional 2020.2.4 (build 86063)
    Teamcity SonarRunner - 2020.1-11.
    Github integration 0.5.91

  • what are you trying to achieve
    Decorating my Pull requests.

  • what have you tried so far to achieve this
    i have followed GitHub Integration | SonarQube Docs
    Able to see pull requests on sonarqube but it is not decorating my github conversation section.
    Created Sonarqube github app account and is installed. Updated Sonarqube global settings with github app information. Created github secrets in repository.

in the Pull requests, i see Last analysis has two warnings, unsure if this is contributing to the decoration failure.

  1. SonarScanner will require Java 11 to run, starting in SonarQube 9.x. Please upgrade the version of Java that executes the scanner and refer to the documentation if needed. - i have Java 11 installed, updated Java_home to the java 11 path and updated wrapper.conf wrapper.java.command=C:\Program Files\Java\jdk-11.0.10\bin\java

  2. Could not find ref ‘master’ in refs/heads, refs/remotes, refs/remotes/upstream or refs/remotes/origin. You may see unexpected issues and changes. Please make sure to fetch this ref before pull request analysis and refer to the documentation.- I have set this to agent side checkout as some forums mentioned it needs a .git file. (i would prefer server side checkout due to our work situation.) - in the .git file, ref/heads there is nothing in here.

Could anyone shed any light on this? anything i should check? review? redo

Hi @pressieguy , welcome to the community!

First of all:

Please upgrade your sonar-scanner, as the latest version is now version 4.6.2.

Once you have done that, scan again. If it does not work, then please share the full debug logs with me here in text format and I will have a look at it.

Best regards,
Daniel

thank you for your suggestion, will try give this a go and get back to you soon.

Hi Daniel,
i ended up using teamcity Sonar Scanner for msbuild to run its scan. that version is 4.8.0.12008.

our sonarqube log is quite huge, did you want me to paste in here directly?

Hi @pressieguy ,

It wasn’t clear from your post what kind of project you were analysing. If that’s a .NET project, indeed you need to use the SonarScanner for .NET. The latest version of this is 5.2.1.

You can upload the log as a text file here. The log must be in debug mode (passing the “-X” flag to the scanner).

Cheers,
Daniel

Hi Daniel,
we have some projects using the old version of sonar scanner and some using SonarScanner for MSBuild 5.0.4 Using the .NET Core version of the Scanner for MSBuild. our code is .net code.

would this be enough logs information to help understand whats really going on to why its not decorating Pull requests ?

ive had to cut out a lot of logs as it was way too big. ive only included the sonar scanner begins and ends

Build_HUB_Pull_Request_3.4.3080.0-PR2318.log (2.8 MB)

Hi @pressieguy ,

I have checked your scanner logs. The second warning you initially mentioned in your post is not there either. Also, there is this warning:

[12:32:08] :	 [Step 9/16] 12:32:08.441 WARN: SCM provider autodetection failed. Please use "sonar.scm.provider" to define SCM of your project, or disable the SCM Sensor in the project settings.

As explained here, this will prevent decoration. You seem to be missing the .git folder in the scanner context.

How are you checking code out in the pipeline? Depending on your approach and as noted in the thread referenced above, you may benefit from specifying the sonar.scm.revision parameter in your analysis.

Build_HUB_Pull_Request.log (40.8 KB)
Hi Daniel
Thanks for the help. weve had to change our checkout code to server side checkout, hence theres no second warning.

im having troubles getting and understanding the sonar.projectbasedir section of the scanner.
i have this in my Sonar begin step.
/d:sonar.scm.provider=git
/d:sonar.projectBaseDir=ReflexHubV3
Our git org is imdex and repo is ReflexHubV3. im following the forums and it continues to fail

[12:15:26] The project base directory doesn’t exist.
[12:15:26] Writing processing summary to :\checkouts\hubv3pullnotests.sonarqube\out\ProjectInfo.log
[12:15:26] Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
[12:15:26]12:15:26.587 Post-processing failed. Exit code: 1
[12:15:26] Process exited with code 1
[12:15:26] Process exited with code 1 (Step: SonarScanner for MSBuild Finish (SonarScanner for MSBuild: finish analysis))
[12:15:26] Step SonarScanner for MSBuild Finish (SonarScanner for MSBuild: finish analysis) failed

Hi @pressieguy ,

From the docs:

sonar.projectBaseDir:
Use this property when you need analysis to take place in a directory other than the one from which it was launched. E.G. analysis begins from jenkins/jobs/myjob/workspace but the files to be analyzed are in ftpdrop/cobol/project1 . The path may be relative or absolute. Specify not the the source directory, but some parent of the source directory. The value specified here becomes the new “analysis directory”, and other paths are then specified as though the analysis were starting from the specified value of sonar.projectBaseDir . Note that the analysis process will need write permissions in this directory; it is where the sonar.working.directory will be created.

I don’t see the use case for you to set this parameter - can you explain why are you setting this?

When you set /d:sonar.projectBaseDir=ReflexHubV3 you are specifying an absolute directory called “ReflexHubV3” that actually does not exist in the execution context. I am not sure which forum threads you are following, but I would recommend to focus on our documentation first.

Cheers,
Daniel

Hi Daniel.
the only reason why i dropped the projectbasedir is because i was getting a
Caused by: Not inside a Git work tree: D:\checkouts\hubv3pullnotests
error. And everyone was suggesting to add the projectbasedir
can sonarqube run with server side checkout? would this be the reason?

INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 6.108s
ERROR: Error during SonarQube Scanner execution
java.lang.IllegalStateException: Unable to load component class org.sonar.scanner.scan.filesystem.ProjectFileIndexer
INFO: Final Memory: 12M/210M
INFO: -----------------------------------------------------------------------
at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:678)
at org.sonar.core.platform.ComponentContainer.getComponentByType(ComponentContainer.java:282)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:352)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:150)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.lang.IllegalStateException: Unable to load component class org.sonar.scanner.scan.filesystem.FileIndexer
at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:632)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:699)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:647)
at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:64)
… 50 more
Caused by: java.lang.IllegalStateException: Unable to load component class org.sonar.scanner.scm.ScmChangedFiles
at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:66)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:632)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.getComponentInstance(AbstractInjectionFactory.java:56)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:699)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:647)
at org.sonar.core.platform.ComponentContainer$ExtendedDefaultPicoContainer.getComponent(ComponentContainer.java:64)
… 64 more
Caused by: Not inside a Git work tree: D:\checkouts\hubv3pullnotests
Java HotSpot™ 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
The SonarQube Scanner did not complete successfully
Process returned exit code 1
.83 Post-processing failed. Exit code: 1
Process exited with code 1
Process exited with code 1 (Step: SonarScanner for MSBuild Finish (SonarScanner for MSBuild: finish analysis))
Step SonarScanner for MSBuild Finish (SonarScanner for MSBuild: finish analysis) failed

found articles like this SonarQube Scanner fails with "Not inside a git worktree" error
How to fix "Not inside a Git work tree" with SonarQube? - Stack Overflow

Hi @pressieguy ,

  1. You are now using server side checkout in your TC pipeline, correct? If this is the case, you will benefit from having a .git folder with all version control metadata so that PR Analysis and other features can work correctly.
  2. It is unclear from your posts what is the situation now: are you setting sonar.projectBaseDir or not? If you are using server side checkout, and the analysis command is fired from the newly checked-out directory containing all sources, you should not need to set this parameter.

Cheers,
Daniel

Hi Daniel.
weve tried different things and managed to get our builds to build using agent side checkout and i have updated a sonar scanner to msbuild.sonarscanner-5.2.1.
we still get the following warnings and i dont know if its related

SonarScanner will require Java 11 to run, starting in SonarQube 9.x. Please upgrade the version of Java that executes the scanner and refer to [the documentation] if needed.

Could not find ref ‘master’ in refs/heads, refs/remotes, refs/remotes/upstream or refs/remotes/origin. You may see unexpected issues and changes. Please make sure to fetch this ref before pull request analysis and refer to [the documentation]

We have Java 11 installed on the server and the agents and set Java_home to the java 11 jdk directory.

This is our sonar begin settings
C:\sonar-scanner-msbuild-5.2.1\SonarScanner.MSBuild.exe begin /k:“org.imdex:reflexhub-v3” /n:“ReflexHubV3” /v:%build.counter% /d:sonar.exclusions=file:\Migrations,\LicenseModel.cs,*.sql,*.c,\libs***.js,*.yaml,file:*.xml,file:*.cshtml,file:***.cpp /d:sonar.github.pullRequest=%teamcity.build.branch% /d:sonar.github.repository=
/d:sonar.github.oauth=
/d:sonar.cfamily.build-wrapper-output.bypass=true /d:sonar.host.url=
/d:sonar.login=
/d:sonar.pullrequest.key=%teamcity.build.branch% /d:sonar.pullrequest.branch=%teamcity.pullRequest.source.branch% /d:sonar.pullrequest.base=%teamcity.pullRequest.target.branch% /d:sonar.verbose=true /d:sonar.scm.provider=git

Running out of ideas what could be the issue.
Build_HUB_Pull_Request (2).log (2.9 MB)

Hi @pressieguy ,

The problem is this one:

[14:09:54] :	 [Step 11/17] 14:09:54.690 WARN: Could not find ref 'master' in refs/heads, refs/remotes, refs/remotes/upstream or refs/remotes/origin

You need to git fetch (docs here) the source and the target branches of the pull request so that they can be located in the refs of your TC pipeline’s local repo. The SonarScanner needs to locate both the source and target branches under either refs/heads, refs/remotes, refs/remotes/upstream or refs/remotes/origin. To debug, once you have done git fetch you may check the refs of your local repo with git show-ref.

Cheers,
Daniel

Hi Daniel,
I managed to get a fetch going aggainst the origin master, however it still doesnt seem to be decorating the github PRs.

there doesnt seem to be any error messages either which makes it bit challenging
Build_Pull_Request_no_tests_22062021.log (2.9 MB)

Any suggestions i could look into.

Hi @pressieguy ,

Everything looks fine to me regarding PR analysis on the SonarScanner side. Decoration is done by the server, after the analysis report has been digested by your server in a Background Task. Are you sure the corresponding Background Task reported no problems?

You would also need to share server-side logs with me, specifically the compute engine ones (ce.log), taken at the time of analysis (2021-06-22 14:17+).

And just to make sure, your server must have network connectivity to your GitHub server. Please, do also share a screenshot of your Administration → ALM Integrations → GitHub page. Both checks should be marked in green. In addition, your project must also be configured to use the GitHub connection (Project Settings → DevOps Platform Integration) - please also share a screenshot of this page showing the green box “Configuration valid”.

Cheers,
Daniel

Hi Daniel.
Thanks for helping me on this issue.
i was looking for logs in the wrong places, i was getting this error in the ce.log.

Caused by: java.lang.IllegalStateException: No token could be found with access to the requested repository with the given application ID and key

All the Github auth tokens, id and secrets were correct. it was the repository identifier that was incorrect. i added my organisation/repository and it worked.
thanks again for your help.