Problem popped up again allowing me to refine the work-around. Good News. I think the problem is resolved via forcing a ‘git fetch’ on the branch when Jenkins builds it.
What worked this time:
- Problem occurred. A build died at the sonarqube quality gate.
- I manually logged into build box, went to workspace for the affected build.
- ran the ‘git merge …’ command - confirmed it indicated an old commitid
- ran a ‘git fetch’ - saw a bunch of stuff come over
- ran the ‘git merge …’ command - confirmed the commitid changed
- restarted the build in Jenkins - this time it passed.
- List item
- Bottom line: the ‘git fetch’ resolves the issue (and I’ll just bake it into our build process)
The longer story, now that I’ve looked at it and my forehead is properly flattened…
- We use Jenkins for our builds (specifically, multi-branch declarative pipeline). When the branch is first created (e.g. in Bitbucket), Jenkins spots it and clones it into a Jenkins workspace and does the first build. No real magic there.
- As additional commits occur on the branch, webhooks (from Bitbucket to Jenkins) notify Jenkins that a new build needs to occur. I believe as part of this, Jenkins identifies the commitid of the new commit.
- Jenkins attempts to be cleaver. It knows it already has a workspace (original clone), and it knows the commitid of the new commit that it needs to build. Apparently, it ‘cheats’ and simply drops the new code into the existing workspace but does not cause the “.git” information to change (i.e. it’s not fetching or pulling, it’s more of a raw update). The build is happy because all the files are precisely what they should be - they match the new commitId.
- But sonar gets handed this lobotomized world. Sonar looks at the ‘.git’ info (which is not updated), calculates the ‘git merge…’ based on stale information, and goes sideways. Adding the ‘git fetch’ to the build (prior to sonar step) insures the ‘.git’ info is properly up to date.
I may not have this perfect, but should be close-enough. The ‘fetch’ is definitely the simple workaround.
Here’s the full details - affected branch is the OG-7522 one
Branch State:
OG-7522: Branch was originally created from 205cfcc. Here’s relevant piece of “Build #1” where it clones. This build passed and was working OK (no changes on the branch yet)
Later (Build #2), we built at commit 2577869. This build failed at the quality gate check. Here’s what the git-retrieval looks like:
Following the failure, I went into the build box, did the ‘git fetch’ (above). Prior to the fetch, the ‘git merge’ command shows 205cf. After the fetch, the ‘git merge’ shows ‘25778’. I then launched a build on Jenkins (Build #3) (also at commit 2577869). The git-retrieval is now:
All of this ‘git retrieval’ (entirety of the log) is coming from the ‘bbs_checkout’ plugin (for pipelines). We are not doing the git commands directly ourselves. I think the ‘bbs_checkout’ is largely identical to the Jenkins ‘git’ plugin (for pipelines)
All that said, I’m not sure what sonarqube (the company/tool) can do about it. Seems like the problem is some shortcuts(?) that jenkins and/or bitbucket has taken. Fix is easy - do a fetch before running sonarqube. (which, at best, is something that could be mentioned in the sonarqube documentation)
Feel free to resolve this ticket as closed 