My repo is hosted on GitHub, I’m using Jenkins for CI.
I do see scan results for ‘master’ in SonarCloud site, and I even see my branches and PRs.
Two issues:
While for ‘master’ I’m able to see scan report and drill down into ‘code’, for branches and PRs there is no any details.
This is definitely related to the fact that I can’t see any modified code in the branches and PRs.
It means that the SonarQube scanner running in Jenkins couldn’t find the master branch (which I’m guessing was the target branch).
This is most likely caused by the way your project is being cloned. Make sure that all branches are cloned (or at least fetch the target branch) and that it’s not a shallow clone.
It works fine when I run sonar-scanner on my local machine. I also have this warning message when running a feature branch on Bitbucket Pipeline. Did we have any solution for it? Where can I put the master (target) branch fetch?
Sorry, I’m very new to this.
I’d like to hear from you soon.
Before running the analysis, you need to run git commands to fetch the target branch. For example if the target of the pull request is master, then you need to do git fetch origin master before running sonar-scanner.
@vitalykarasik Hm, I don’t see how that’s possible… Is this a reproducible issue? If you run the analysis again, do you see the warning, when in fact the Jenkins workspace has the ref? (Are you looking at the correct workspace?)
Not much luck. I run git fetch origin master right before running the sonar-scanner. and still face the issue. Below is the log on bitbucket pipeline build
`========git fetch origin master
branch master -> FETCH_HEAD
========Run scanner
INFO: Scanner configuration file: /cicd/sonar-scanner-3.3.0.1492/conf/sonar-scanner.properties
INFO: Project root configuration file: NONE
INFO: SonarQube Scanner 3.3.0.1492
INFO: Java 1.8.0_181 Oracle Corporation (64-bit)
INFO: Linux 4.14.84-coreos amd64
INFO: Bitbucket Cloud Pipelines detected
INFO: User cache: /root/.sonar/cache
INFO: SonarQube server 7.7.0
INFO: Default locale: “en”, source code encoding: “UTF-8” (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=578ms
INFO: Server id: BD367519-AWHW8ct9-T_TB3XqouNu
INFO: User cache: /root/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=126ms
INFO: Load/download plugins (done) | time=26582ms
INFO: Loaded core extensions: branch-scanner
INFO: Process project properties
INFO: Execute project builders
INFO: Execute project builders (done) | time=6ms
INFO: Project key: franklinpython
INFO: Base dir: /opt/atlassian/pipelines/agent/build
INFO: Working dir: /opt/atlassian/pipelines/agent/build/.scannerwork
INFO: Load project settings
INFO: Load project settings (done) | time=113ms
INFO: Load project branches
INFO: Load project branches (done) | time=107ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=96ms
INFO: Load branch configuration
INFO: Load branch configuration (done) | time=3ms
INFO: Load project repositories
INFO: Load project repositories (done) | time=145ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=121ms
INFO: Load active rules
INFO: Load active rules (done) | time=2599ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=102ms
INFO: Organization key: ansarada
INFO: Branch name: test-branch, type: short living
INFO: SCM collecting changed files in the branch
WARN: Could not find ref: master in refs/heads or refs/remotes/origin
INFO: SCM collecting changed files in the branch (done) | time=69ms
INFO: Indexing files…
INFO: Project configuration:
INFO: 4 files indexed
INFO: Quality profile for py: Sonar way
INFO: ------------- Run sensors on module franklinpython
INFO: Sensor JavaXmlSensor [java]
INFO: Sensor JavaXmlSensor [java] (done) | time=3ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=14ms
INFO: Sensor Python Squid Sensor [python]
WARN: Metric ‘comment_lines_data’ is deprecated. Provided value is ignored.
INFO: Python test coverage
INFO: Parsing report ‘/opt/atlassian/pipelines/agent/build/coverage.xml’
INFO: Sensor Python Squid Sensor [python] (done) | time=206ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=3ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=11ms
INFO: Sensor JavaSecuritySensor [security]
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/java
INFO: 00:11:35.974 Building Type propagation graph
INFO: 00:11:35.979 Running Tarjan on 0 nodes
INFO: 00:11:35.979 Tarjan found 0 components
INFO: 00:11:35.979 Variable type analysis: done
INFO: UCFGs: 0, excluded: 0, source entrypoints: 0
INFO: No UCFGs have been included for analysis.
INFO: Sensor JavaSecuritySensor [security] (done) | time=12ms
INFO: Sensor CSharpSecuritySensor [security]
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/ucfg_cs2
INFO: 00:11:35.982 Building Type propagation graph
INFO: 00:11:35.982 Running Tarjan on 0 nodes
INFO: 00:11:35.982 Tarjan found 0 components
INFO: 00:11:35.982 Variable type analysis: done
INFO: UCFGs: 0, excluded: 0, source entrypoints: 0
INFO: No UCFGs have been included for analysis.
INFO: Sensor CSharpSecuritySensor [security] (done) | time=0ms
INFO: ------------- Run sensors on project
INFO: 1 file had no CPD blocks
INFO: Calculating CPD for 1 file
INFO: CPD calculation finished
INFO: SCM writing changed lines
**********WARN: Could not find ref: master in refs/heads or refs/remotes/origin
INFO: SCM writing changed lines (done) | time=2ms
INFO: Analysis report generated in 371ms, dir size=87 KB
INFO: Analysis report compressed in 7ms, zip size=19 KB
INFO: Analysis report uploaded in 195ms
INFO: ANALYSIS SUCCESSFUL, you can browse https://sonarcloud.io/dashboard?id=franklinpython&branch=test-branch&resolved=false
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=AWi6_vDNSr1wYhq3igZU
INFO: Analysis total time: 7.956 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 39.716s
INFO: Final Memory: 36M/659M
INFO: ------------------------------------------------------------------------
t`
@janos - thank you again.
A ran some additional tests and learnt the following:
We don’t have to use -Dsonar.branch.target=master - it’s default
When there is no diff between branch/PR and master, SonarCloud portal presents empty “Issues” tab, it confused me
So my status for now - SonarCloud does analyses commits to branches/PRs, submitted by Jenkins CI.
But - GitHub status is not updated, even “See the PR” link in Sonar points to the proper GitHub PR page.
I installed Sonar app on GitHub.
I think there is a bit of confusion here. Let’s try to clear that up.
In your local experiment with Gradle, using -Dsonar.branch.name= is not relevant for debugging your issue. This parameter is used for branch analysis, not useful for pull requests. And, you mention that you don’t have to use -Dsonar.branch.target=master because master is the default. That’s not really accurate. In the current implementation the scanner requires that you pass this parameter, even if the value is the same as the default on the server, otherwise it may not compute changed lines correctly.
In any case, the previous paragraph is not really important for fixing Pull Request analysis. You need to provide the following parameters:
sonar.pullrequest.branch - the name of the branch in this pull request
sonar.pullrequest.base - the name of the branch you want to merge into
sonar.pullrequest.key - the pull request number, for example 1234
sonar.pullrequest.provider=github
sonar.pullrequest.github.repository - the “slug” of your repository on GitHub, for example orgname/reponame
And you need to make sure the target branch (the value of sonar.pullrequest.base) is fetched.
Now I’m testing from Jenkins CI, using “-Dsonar.branch.name=$GIT_BRANCH” for branch analyze and “-Dsonar.pullrequest.branch=$GIT_BRANCH -Dsonar.pullrequest.key=$CHANGE_ID” for PR analyze.
In my sonar.gradle I have property “sonar.projectKey” and “sonar.organization”.
I’m testing PR analyze now. I do see changes line on my code, and SonarCloud portal shows proper link to my GitHub PR and “for merge with master”.
I added “-Dsonar.pullrequest.provider=github -Dsonar.pullrequest.base=master” and “sonar.pullrequest.github.repository”, it didn’t improved situation - on my GitHub PR page I do see Jenkins status, but no SonarCloud one.
As I mentioned already, don’t use -Dsonar.branch.name in a Pull Request analysis. It’s pointless and potentially confusing.
I suggest to first try to get PR analysis working correctly locally, not from Jenkins. Once you manage to get that working, then you should be able to port that to Jenkins. Without knowing the actual values of $GIT_BRANCH and $CHANGE_ID, and without knowing the specific PR in question on GitHub, I have no way to verify that your setup is correct. Is the project publicly visible? If yes, you could share it, and the exact command that you are running. Then I may be able to pinpoint the problem.
My real command is
./gradlew sonarqube -Dsonar.pullrequest.branch=PR-288 -Dsonar.pullrequest.key=288 -Dsonar.pullrequest.provider=github -Dsonar.pullrequest.base=master -Dsonar.pullrequest.github.repository=Company/project
Is there a chance that I should configure some GitHub hook manually? As I wrote, in SonarCloud portal I do see right URL of my PR.
Unfortunately, no. I’ll be glad to share credentials in private. Or - I’ll try to reproduce the issue with some opensource project fork.
PS:
IMHO:
While SonarSource core is very strong and useful product, its integration is confusing. I understand that there are many plugins for different build tools (Gradle, Maven, …), CIs (Travis, Jenkins, …) and version-control systems. While there are blog posts and forum threads, I see no full “User manual”, updated for current state of SonarSource ecosystem.
“I’m not sure if it could be a problem, but how our jenkins build works is it merges master into the branch and creates a new branch named PR- eg PR-123. I think this is standard jenkins behaviour though, and we don’t build the actual branch during pull request as we don’t care about it. But I guess github wont have the same sha1?”
