Bitbucket Server + Jenkins Pull Request Decoration

[obi (Phil C)]

Software:

• SonarQube version 9.2.4 (build 50792) Developer Edition, running in Centos 7 VM
• Jenkins 2.319.1 LTS
  • Bitbucket Branch Source plugin 737.vdf9dc06105be
  • SonarQube Scanner for Jenkins 2.14
• SonarScanner 4.6.2.2472
• Bitbucket Server v7.19.1

I am unable to get my Bitbucket Pull Requests decorated with SonarQube reports.

I have:

• Followed Pull Request Analysis | SonarQube Docs
• Followed Bitbucket Server Integration | SonarQube Docs
• In SonarQube, I can see more than one Branch in sonarqube, but no pull requests. I have multiple successful analysis on the source branch. There are no warnings against the build analysis
• In ‘Administration’>>‘DevOps Platform Integration’ in the Bitbucket section, I have green checkmarks against both ‘Quality Gate status reporting’ and ‘Import repositories from your DevOps platform’
• In ‘Project Settings’>>‘DevOps Platform Integration’ I get ‘Configuration Valid’ when I press ‘Check Configuration’
• My Jenkins job is a Multibranch Pipeline, and my Jenkinsfile includes a withSonarQubeEnv(installationName: 'SonarQube') section, the Jenkins build is successfully posting the results to SonarQube, and I can browse the correct branch in SonarQube and see the results
• I modified my SonarScanner command line invocation to include the ‘-X’ flag and I can see this:

09:43:31.246 INFO: Load project settings for component key: 'PROJ_projname'
09:43:31.266 DEBUG: GET 200 http://sonarqube.ldevel.net:9000/api/settings/values.protobuf?component=PROJ_projname | time=20ms
09:43:31.268 INFO: Load project settings for component key: 'PROJ_projname' (done) | time=22ms
09:43:31.282 INFO: Load project branches
09:43:31.290 DEBUG: GET 200 http://sonarqube.ldevel.net:9000/api/project_branches/list?project=PROJ_projname | time=8ms
09:43:31.294 INFO: Load project branches (done) | time=12ms
09:43:31.295 INFO: Load project pull requests
09:43:31.302 DEBUG: GET 200 http://sonarqube.ldevel.net:9000/api/project_pull_requests/list?project=PROJ_projname | time=6ms
09:43:31.305 INFO: Load project pull requests (done) | time=10ms
09:43:31.305 INFO: Load branch configuration
09:43:31.306 INFO: Detected branch/PR in 'Jenkins'
09:43:31.306 INFO: Auto-configuring branch 'feature/PRJ-112-0.2.0-integrate-sonarqube'
09:43:31.307 INFO: Load branch configuration (done) | time=2ms

• This topic Sonarqube Pull Request Decoration - Analyses parameters - Get help / SonarQube - SonarSource Community states that the ‘Discover pull requests from origin’ setting should be set in the Bitbucket Branch Source plugin, but that must be for a previous version, as it’s not present on my installation. I have the ‘Discover Branches’ setting checked, but the only other ‘Discover…’ options available are ‘Discover other refs’ and ‘Discover tags’

Please can you assist me in making progress?

[ganncamp (G Ann Campbell)]

Hi,

Since you’re using the latest version of SonarScanner for Jenkins, you should be seeing the fact that it’s a PR analysis picked up automatically from your environment. However, what we see from your logs is that it’s being treated as a branch analysis, not a PR analysis.

@obi

09:43:31.306 INFO: Auto-configuring branch 'feature/PRJ-112-0.2.0-integrate-sonarqube'

So I have to start with the dumb questions: the developer actually created a PR, right? And that PR is actually what triggered the analysis and is being analyzed?

 
Ann

[obi (Phil C)]

Hi Ann,

Thanks for your reply. I have definitely created a PR, and bitbucket is showing a build status against it.

The Jenkins build is against the branch, the PR isn’t shown in Jenkins at all. I don’t know if it’s because of the missing ‘Discover pull requests from origin’ setting in the Bitbucket Branch Source plugin. I found something online (can’t find the page now) saying I should add pull-requests/*/from to Discover other refs in the Branch Source configuration, and now I have a separate job in Jenkins for the PR. This means I’d now need two separate builds, one for the branch and one for the PR. Additionally the PR build doesn’t build automatically when I do a new push to the branch (which is the source branch for the PR), for reasons I’ve yet to discover. Is this really the way it’s meant to be set up?

When the Jenkins build for the PR finally completed, there was this in the log:

10:55:19.372 INFO: Auto-configuring branch 'pull-requests-24'

So Jenkins was still treating it as a branch rather than a PR, and SonarQube still does not have any awareness of any PRs on its GUI. Not sure where to go from here, any other ideas?

Thanks,
Phil

[ganncamp (G Ann Campbell)]

Hi Phil,

We did some work in the SonarScanner for Jenkins in the 8-series to get this all picked up automatically. I don’t think you need the Bitbucket Branch Source plugin (which isn’t maintained by us, so I don’t know anything about it). But I guess it won’t interfere…?

The Scanner for Jenkins is looking for two environment variables to know it’s a PR; $CHANGE_ID and $CHANGE_TARGET. And then it sets parameters sonar.pullrequet.key, sonar.pullrequest.branch, and sonar.pullrequest.base. So, can you check your environment to make sure those variables are set? Can you add -Dsonar.scanner.dumpToFile=[file path] to see what analysis parameters the scanner is working with?

 
Ann

[obi (Phil C)]

Hi Ann,

I’ve done another run with dumpToFile, and none of the sonar.pullrequest variables are set. I’m not sure Jenkins knows about the PR, because it’s building from the branch. I thought of using the Bitbucket REST API to get the PR info, but don’t know how to give Jenkins the credentials for it (even though it should already have them somewhere) , and that would have to be combined with with using git to get the PRs for the commit.

Any ideas how we can get Jenkins the info?
Thanks,
Phil

[ganncamp (G Ann Campbell)]

Hi Phil,

I was wrong.
Our docs do say you need to have the correct flavor of Branch Source plugin installed in Jenkins. (Altho it’s still not our plugin. )

They also say (and I think this must be where the env vars come from):

The automatic configuration of branches and Pull Requests relies on environment variables available in Multibranch Pipeline jobs.

Aand those env vars are supplied by the branch source plugin. Which is why you need it in there.

Digging around a little, I see this in the BitBucket Branch Source plugin docs:

2. For Jenkins Pipeline jobs to function for pull requests on Bitbucket Server 7.x, you need to enable “Call Changes api” option in the plugin configuration.
   a. Bitbucket Server 7.x no longer automatically creates the required refs for pull requests. Calls to this new API are required to trigger ref creation.

This is a change in Bitbucket Server 7.x.

So, do you have that turned on?

 
Ann

[obi (Phil C)]

Hi Ann,

Apologies, I should have mentioned that earlier; yes I found the same pages and have had ‘Call Changes API’ enabled from the start. Sorry we keep running up against issues!

Any other ideas?
Thanks,
Phil

[ganncamp (G Ann Campbell)]

Hi Phil,

So… all the pieces seem to be in place, in all the right versions. But we don’t have the environment vars that will trigger PR (versus branch) analysis.

I think it’s time to see what env vars we do have. Can you add a printenv to your build script? Maybe in a couple places just to see if the list / values fluctuate?

Also, and since you seem to have already crossed all your t’s this feels like a silly question, but. You did this part, right?

You need to configure your Multibranch Pipeline job correctly to avoid issues with Pull Request decoration. From your Multibranch Pipeline job in Jenkins, go to Configure > Branch Sources > Behaviors .

For Bitbucket and GitHub, under Discover pull requests from origin , make sure The current pull request revision is selected.

 
Ann

[obi (Phil C)]

Hi Ann, thanks for your continued help.

One thing that’s occurred to me that is a little non-standard in our Jenkins build is that we’re building in a docker agent:

		stage('Build') {
			agent {
				docker {
					image 'mybuild:centos7.9.2009'
					args "-u root:root -e HOSTUID=$HOSTUID"
					reuseNode true
				}
			}
			steps {
                            sh 'printenv'
                            ...

But I inserted a printenv at the start and end of the Build stage, and added a new stage before the Build stage that just does a sh 'printenv', and in neither of them could I see any CHANGE variables. Here’s the complete list of available variables (with their values stripped):

_
BRANCH_NAME
BUILD_DISPLAY_NAME
BUILD_ID
BUILD_NUMBER
BUILD_TAG
BUILD_URL
CI
EXECUTOR_NUMBER
GIT_BRANCH
GIT_COMMIT
GIT_LOCAL_BRANCH
GIT_PREVIOUS_COMMIT
GIT_PREVIOUS_SUCCESSFUL_COMMIT
GIT_URL
HOME
HOSTUID
HUDSON_COOKIE
HUDSON_HOME
HUDSON_SERVER_COOKIE
HUDSON_URL
JENKINS_HOME
JENKINS_NODE_COOKIE
JENKINS_SERVER_COOKIE
JENKINS_URL
JOB_BASE_NAME
JOB_DISPLAY_URL
JOB_NAME
JOB_URL
LANG
LOGNAME
NODE_LABELS
NODE_NAME
PATH
PWD
RUN_ARTIFACTS_DISPLAY_URL
RUN_CHANGES_DISPLAY_URL
RUN_DISPLAY_URL
RUN_TESTS_DISPLAY_URL
SHELL
SHLVL
STAGE_NAME
USER
WORKSPACE_TMP
WORKSPACE

Within Docker all the variables were the same (with trivial differences to e.g. HOME) except that the following variables were no longer available:

HUDSON_COOKIE
LANG
LOGNAME
STAGE_NAME
USER

But I’m not convinced that these could affect the behaviour.

On that last point, as I’ve mentioned before my Jenkins Bitbucket Source Plugin doesn’t have the ‘Discover pull requests from Origin’ option. I’ve added a small screenshot so you can see I’m not going crazy!

Cheers,
Phil

[ganncamp (G Ann Campbell)]

Hi Phil,

I doubt this is about Docker. That option’s supposed to be under Configure > Branch Sources > Behaviors. Is that where that screenshot’s from?

Ann

[obi (Phil C)]

Hi Ann,

Thanks. Yes, it’s from Configure > Branch Sources > Behaviors. All the sources I’ve found on various forums seem to expect it to be there too, but it just isn’t.

Thanks,
Phil

[ganncamp (G Ann Campbell)]

Hi Phil,

I hate to say this, but I think this is going to be on the Jenkins side. The env vars we need to run it as a PR analysis are missing, and you can’t find the option to make them available. I’ve done a little digging online & I’m not finding anything obvious. All your versions are reasonably up to date, so it’s not likely to be that you’re missing an upgrade to make this appear.

All I can think is to check your browser console for errors when you should be seeing this “pull requests from origin” option and you’re not, and to see what other Jenkins plugins are contributing to that dropdown that might be interfering.

 
Ann

[obi (Phil C)]

Hi Ann,

Thanks for all your help, I’ll have to look at creating a post on the Jenkins forum to see if those with more experience with Jenkins to think about.

Thanks again for your persistence!
Phil

[regicsolutions]

Hi @obi wondering if you ever resolved this issue? I’m also having trouble getting PR decoration working with Bitbucket Server

[obi (Phil C)]

Hi @regicsolutions, unfortunately I had to give up because of other priorities, so I never got it working.

Apologies,
Phil