We are evaluating self hosted SonarQube 7.7 DeveloperEdition
Running PullRequest Analysis on Jenkins with Maven (sonar.scanner.appVersion=3.5.0.1254/3.6.0)
Analysis runs well, but PullRequest Decoration won’t work. We have two warnings:
Failed to create Bitbucket Server Quality Report for commit SHA ‘…’
Could not find ref ‘master’ in refs/heads or refs/remotes/origin. You may see unexpected issues and changes. Please make sure to fetch this ref before pull request analysis.
About the first warning
The listed SHA does not exists as we configured Jenkins, to build the PR branch with merged to target branch. Is there a possibility to give the correct SHA (revision from PR) via commandline?
Could not find ref ‘master’ in refs/heads or refs/remotes/origin. You may see unexpected issues and changes. Please make sure to fetch this ref before pull request analysis.
as expected, this has no effect on the described problem
even if the sha exists and is the commit at the tip of the PullRequest, the warning Failed to create Bitbucket Server Quality Report for commit SHA ‘…’ occurs and no PR-decoration happended.
so how could I find out, if pull-request decoration generally works against our BitBucket server.
Could I somehow increase log levels around the warning message which I cited above?
We are also having both of these problems and I am curious about a solution. I have been able to identify something that may be contributing to the issue, but I have not yet found a solution:
Jenkins BitBucket Server integration enables “PR discovery” in Jenkins, which creates jenkins specific PR “branches”. These are virtual branches in that they only exist for the life of the PR and only inside of Jenkins. The byproduct of this is that Jenkins will, under the hood, do some merging shenanigans some times that will change the commit ID. For example, here is a recent error we got:
Failed to create Bitbucket Server Quality Report for commit SHA '0922d2f...'
The PR has only one commit which is 366e037906c...
If I look at the Jenkins logs, I can see how Sonar is picking up the 0922d2f... commit from this part of the Jenkins PR build log:
As you can see, it does some git shenanigans to create a PR specific commit based on the commit from the actual PR for execution in Jenkins. This appears to be what is causing the SHA disconnect, at least from what I can see. Unfortunately, I don’t have a solution (I am taking ideas!).
Separately, we are also seeing the other warning, which we also don’t have a solution for, so would love some feedback there as well.
Could not find ref ‘master’ in refs/heads or refs/remotes/origin. You may see unexpected issues and changes. Please make sure to fetch this ref before pull request analysis.
against Could not find ref ‘master’ in refs/heads or refs/remotes/origin. You may see unexpected issues and changes. Please make sure to fetch this ref before pull request analysis.:
it really helped to just give Sonar-Scanner what it wants: a reference at refs/remotes/origin/master
in details: this should be the branch, where your PullRequest is targeted at. For SonarQube this is the (long running) branch, it compares the findings/false-positives etc. luckily Jenkins offers some env variables for this
we solved this via: git fetch ${repo.sshUrl} +refs/heads/${env.CHANGE_TARGET}:refs/remotes/origin/${env.CHANGE_TARGET}
this worked well - the warning is not shown anymore
against: Failed to create Bitbucket Server Quality Report for commit SHA '0922d2f...'
(lets stay with your abbreviated git SHAs for the moment)
from what I experiences, Sonar-Scanner is just looking for what is written in HEAD
so although Sonar-Scanner is analysing the merged code (represented by SHA 9922d2f...) we faked just HEAD, so it points to SHA 366e037906c... (the tip of PR-73)
done this via git reset --soft refs/remotes/origin/${env.BRANCH_NAME}
BUT this lead to the similar warning message: Failed to create Bitbucket Server Quality Report for commit SHA '366e037906c...' … correct SHA but still not decorating the PullRequest at BitBucket (Server)
As @barclay-reg mentioned, to fix the first issue, you should ensure we can compute the diff between PR branch and target/base. So you have to fetch it.
For the second issue, this is a requirement of the Bitbucket Server API to pass the HEAD commit of the PR when publishing a report. By default we get this commit sha1 by simply looking t the current HEAD of the local repo. If you are doing some SCM operations that are modifying the local repo, you can try to pass manually: -Dsonar.pullrequest.bitbucketserver.headSha=xxxxx to the scanner.
For Jenkins, this is probably something like: -Dsonar.pullrequest.bitbucketserver.headSha=${env.GIT_COMMIT}
many thanks for this hint - this makes it more easier.
Currently I found the point why PR decoration not worked for me: we have a BitBucket version which has no support for BitBucket Code Insights. This required BitBucket feature was added with BB 5.15 - so I recomment a hint within sonarqube docu - maybe also about sonar.pullrequest.bitbucketserver.headSha
Thank you both so much, this information is incredibly helpful for context, although I am still running into issues, and I wonder if this is because we are doing this through gradle? Here is what I am doing in Jenkins right now, and I am still getting both warnings:
For the commit issue, it looks like GIT_COMMIT is returning the commit that jenkins creates (in the merge strategy definition for the bitbucket plugin), which should be correct now that we have changed our BitBucket plugin discovery strategy, it is always returning the latest commit of the PR itself, in this case 366e037906c..., and yet that one is still resulting in failure. Are there other requirements for that commit other than that it is the HEAD of the PR itself? Is there some target branch requirements that I am missing?
@barclay-reg if you could perhaps give a sample jenkins stage snippet that you got working, that would be really helpful. I am doing a lot of trial and error right now and I am not sure how much closer I am getting to success at this point
UPDATE! This appears to be the following issue: Failed to create Bitbucket Server Quality Report - HTTP 400 - annotations required
We are seeing 400 errors with bitbucket complaining about the annotations (I was finally able to get those logs). We are going to try updating to sonar 7.8 and see if that resolves it, and will report back here with results.
Thank you @ganncamp, I will look out for it once it is available! @Julien_HENRY, for now, I am at a total loss to get this last issue resolved so we can get consistent PR decoration. Here is the current state of our affairs, if there is something obvious I am missing that you can see that will resolve this, you will be my hero!
BitBucket Server version: 5.16.3
SonarQube version: 7.7
PR Commit ID: 18c4358c…
PR Number: 75
PR Branch: PR-75 (this comes from the Jenkins BitBucket integration, since BitBucket server creates ephemeral branches when PRs are submitted).
Actual Branch that PR comes from: begoldsm/my-feature
Jenkins File Snippet for PR:
if (env.CHANGE_ID) {
withCredentials(...) {
status = sh returnStatus: true,
script: '''#!/bin/bash
git fetch $repoUrl +refs/heads/$CHANGE_TARGET:refs/remotes/origin/$CHANGE_TARGET
# also tried with git reset --soft refs/remotes/origin/$BRANCH_NAME when using the merge to target strategy in jenkins
./gradlew check sonarqube --info \
-Dsonar.host.url=$SONAR_HOST \
-Dsonar.login=$SONAR_TOKEN \
-Dsonar.pullrequest.branch=$BRANCH_NAME \ # Also tried the feature branch name
-Dsonar.pullrequest.key=$CHANGE_ID \
-Dsonar.pullrequest.bitbucketserver.headSha=18c4358c... # I explicitly hard coded it for the run to see if that works.
'''
}
}
This results in the following:
All iterations result in the “error” warning (shown below), however the first run does create the SonarQube insights cog in the PR overview (which I can click on and get information about).
All subsequent commits or runs through sonar do not update that insights cog and they all end up with the following warning from sonar: Failed to create Bitbucket Server Quality Report for commit SHA '18c4358c...'
Note that I am quite happy that we resolved the other warning with the git fetch, which is great! Now we just need to figure out why we are still getting this quality report warning.
Yup, we are doing that across the board now in our Jenkins flows. The warning now is a false warning for some of our tests, since the commits we are using to test don’t have any anotations (just a jenkinsfile update), so it sends the POST to bitbucket with an empty body, which results in the 400 error that then gets reported back to Sonar as the warning for the SHA. The reality is that it creates the insights on the first run, though, and then subsequent runs don’t get an update because there isn’t anything to update. So, in this case, the warning is a little misleading. It appears that the issue is already fixed (and validated by you @Julien_HENRY! ) and will be ready when Sonar 7.8 releases: https://jira.sonarsource.com/browse/SONAR-11877
This works for me to determine the SHA to be set with the mentioned properties: def baseCommit = bat(script: "git rev-parse origin/" + env.BRANCH_NAME, returnStdout: true).trim()
