SonarCloud doesn't update GitHub status


(Vitaly Karasik) #1

My repo is hosted on GitHub, I’m using Jenkins for CI.
I do see scan results for ‘master’ in SonarCloud site, and I even see my branches and PRs.
Two issues:

  1. While for ‘master’ I’m able to see scan report and drill down into ‘code’, for branches and PRs there is no any details.

And “Last analysis” for my project is not updated, i.e. it’s Jan, 29 instead of Jan,30.

There is “Task :sonarqubeCould: not find ref: master in refs/heads or refs/remotes/origin” warning in Jenkins log, I have no idea if it related.

  1. Even I install SonarQube GitHub application, there is no Sonar status in GitHub.

Solution for the second issue was modify GitHub Jenkins plugin branch discovery strategy - see my comment from Feb,14.


SonarCloud now not updating GitHub PR and Checks
(Duarte Meneses) #2

This is definitely related to the fact that I can’t see any modified code in the branches and PRs.
It means that the SonarQube scanner running in Jenkins couldn’t find the master branch (which I’m guessing was the target branch).
This is most likely caused by the way your project is being cloned. Make sure that all branches are cloned (or at least fetch the target branch) and that it’s not a shallow clone.


(Franklin) #3

It works fine when I run sonar-scanner on my local machine. I also have this warning message when running a feature branch on Bitbucket Pipeline. Did we have any solution for it? Where can I put the master (target) branch fetch?
Sorry, I’m very new to this.
I’d like to hear from you soon.


(Janos Gyerik) #4

Before running the analysis, you need to run git commands to fetch the target branch. For example if the target of the pull request is master, then you need to do git fetch origin master before running sonar-scanner.


(Vitaly Karasik) #5

@dmeneses, @janos - thank you.
As far as I see, Jenkins workspace does have ‘master’ ref:

cat .git/refs/remotes/origin/master
18f0…361


(Janos Gyerik) #6

@vitalykarasik Hm, I don’t see how that’s possible… Is this a reproducible issue? If you run the analysis again, do you see the warning, when in fact the Jenkins workspace has the ref? (Are you looking at the correct workspace?)


(Vitaly Karasik) #7

@janos - Yes, as far as I understand, it’s the actual Jenkins workspace.

Because Jenkins checkout parameters are tricky, I decided to run a manual test right now:

  1. I cloned my repo
  2. I ran “./gradlew sonarqube” - SonarCloud was updated for ‘master’
  3. I ran “git checkout -b vitaly_branch_sonar”, touch some files, “git commit -m “Sonar tests””
  4. ./gradlew sonarqube -Dsonar.branch.name=vitaly_branch_sonar

Result: I do see “vitaly_branch_sonar” in SonarCloud portal, but without scan report - the same issue as for Jenkins.


(Franklin) #8

Thanks Janos, I’ll try adding the git fetch somewhere in our bitbucket pipeline.

Thanks again


(Franklin) #9

Not much luck. I run git fetch origin master right before running the sonar-scanner. and still face the issue. Below is the log on bitbucket pipeline build

`========git fetch origin master

  • branch master -> FETCH_HEAD
    ========Run scanner
    INFO: Scanner configuration file: /cicd/sonar-scanner-3.3.0.1492/conf/sonar-scanner.properties
    INFO: Project root configuration file: NONE
    INFO: SonarQube Scanner 3.3.0.1492
    INFO: Java 1.8.0_181 Oracle Corporation (64-bit)
    INFO: Linux 4.14.84-coreos amd64
    INFO: Bitbucket Cloud Pipelines detected
    INFO: User cache: /root/.sonar/cache
    INFO: SonarQube server 7.7.0
    INFO: Default locale: “en”, source code encoding: “UTF-8” (analysis is platform dependent)
    INFO: Load global settings
    INFO: Load global settings (done) | time=578ms
    INFO: Server id: BD367519-AWHW8ct9-T_TB3XqouNu
    INFO: User cache: /root/.sonar/cache
    INFO: Load/download plugins
    INFO: Load plugins index
    INFO: Load plugins index (done) | time=126ms
    INFO: Load/download plugins (done) | time=26582ms
    INFO: Loaded core extensions: branch-scanner
    INFO: Process project properties
    INFO: Execute project builders
    INFO: Execute project builders (done) | time=6ms
    INFO: Project key: franklinpython
    INFO: Base dir: /opt/atlassian/pipelines/agent/build
    INFO: Working dir: /opt/atlassian/pipelines/agent/build/.scannerwork
    INFO: Load project settings
    INFO: Load project settings (done) | time=113ms
    INFO: Load project branches
    INFO: Load project branches (done) | time=107ms
    INFO: Load project pull requests
    INFO: Load project pull requests (done) | time=96ms
    INFO: Load branch configuration
    INFO: Load branch configuration (done) | time=3ms
    INFO: Load project repositories
    INFO: Load project repositories (done) | time=145ms
    INFO: Load quality profiles
    INFO: Load quality profiles (done) | time=121ms
    INFO: Load active rules
    INFO: Load active rules (done) | time=2599ms
    INFO: Load metrics repository
    INFO: Load metrics repository (done) | time=102ms
    INFO: Organization key: ansarada
    INFO: Branch name: test-branch, type: short living
    INFO: SCM collecting changed files in the branch
    WARN: Could not find ref: master in refs/heads or refs/remotes/origin
    INFO: SCM collecting changed files in the branch (done) | time=69ms
    INFO: Indexing files…
    INFO: Project configuration:
    INFO: 4 files indexed
    INFO: Quality profile for py: Sonar way
    INFO: ------------- Run sensors on module franklinpython
    INFO: Sensor JavaXmlSensor [java]
    INFO: Sensor JavaXmlSensor [java] (done) | time=3ms
    INFO: Sensor HTML [web]
    INFO: Sensor HTML [web] (done) | time=14ms
    INFO: Sensor Python Squid Sensor [python]
    WARN: Metric ‘comment_lines_data’ is deprecated. Provided value is ignored.
    INFO: Python test coverage
    INFO: Parsing report ‘/opt/atlassian/pipelines/agent/build/coverage.xml’
    INFO: Sensor Python Squid Sensor [python] (done) | time=206ms
    INFO: Sensor PythonXUnitSensor [python]
    INFO: Sensor PythonXUnitSensor [python] (done) | time=3ms
    INFO: Sensor JaCoCo XML Report Importer [jacoco]
    INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3ms
    INFO: Sensor Zero Coverage Sensor
    INFO: Sensor Zero Coverage Sensor (done) | time=11ms
    INFO: Sensor JavaSecuritySensor [security]
    INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/java
    INFO: 00:11:35.974 Building Type propagation graph
    INFO: 00:11:35.979 Running Tarjan on 0 nodes
    INFO: 00:11:35.979 Tarjan found 0 components
    INFO: 00:11:35.979 Variable type analysis: done
    INFO: UCFGs: 0, excluded: 0, source entrypoints: 0
    INFO: No UCFGs have been included for analysis.
    INFO: Sensor JavaSecuritySensor [security] (done) | time=12ms
    INFO: Sensor CSharpSecuritySensor [security]
    INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/ucfg_cs2
    INFO: 00:11:35.982 Building Type propagation graph
    INFO: 00:11:35.982 Running Tarjan on 0 nodes
    INFO: 00:11:35.982 Tarjan found 0 components
    INFO: 00:11:35.982 Variable type analysis: done
    INFO: UCFGs: 0, excluded: 0, source entrypoints: 0
    INFO: No UCFGs have been included for analysis.
    INFO: Sensor CSharpSecuritySensor [security] (done) | time=0ms
    INFO: ------------- Run sensors on project
    INFO: 1 file had no CPD blocks
    INFO: Calculating CPD for 1 file
    INFO: CPD calculation finished
    INFO: SCM writing changed lines
    **********WARN: Could not find ref: master in refs/heads or refs/remotes/origin
    INFO: SCM writing changed lines (done) | time=2ms
    INFO: Analysis report generated in 371ms, dir size=87 KB
    INFO: Analysis report compressed in 7ms, zip size=19 KB
    INFO: Analysis report uploaded in 195ms
    INFO: ANALYSIS SUCCESSFUL, you can browse https://sonarcloud.io/dashboard?id=franklinpython&branch=test-branch&resolved=false
    INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
    INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=AWi6_vDNSr1wYhq3igZU
    INFO: Analysis total time: 7.956 s
    INFO: ------------------------------------------------------------------------
    INFO: EXECUTION SUCCESS
    INFO: ------------------------------------------------------------------------
    INFO: Total time: 39.716s
    INFO: Final Memory: 36M/659M
    INFO: ------------------------------------------------------------------------
    t`

(Janos Gyerik) #10

You also need to specify -Dsonar.branch.target to ensure that the changed files are computed correctly.

And when you do this, I expect you will not get the warning anymore about "Could: not find ref: …”


(Vitaly Karasik) #11

@janos - thank you again.
A ran some additional tests and learnt the following:

  • We don’t have to use -Dsonar.branch.target=master - it’s default
  • When there is no diff between branch/PR and master, SonarCloud portal presents empty “Issues” tab, it confused me

So my status for now - SonarCloud does analyses commits to branches/PRs, submitted by Jenkins CI.
But - GitHub status is not updated, even “See the PR” link in Sonar points to the proper GitHub PR page.
I installed Sonar app on GitHub.


(Janos Gyerik) #12

I think there is a bit of confusion here. Let’s try to clear that up.

In your local experiment with Gradle, using -Dsonar.branch.name= is not relevant for debugging your issue. This parameter is used for branch analysis, not useful for pull requests. And, you mention that you don’t have to use -Dsonar.branch.target=master because master is the default. That’s not really accurate. In the current implementation the scanner requires that you pass this parameter, even if the value is the same as the default on the server, otherwise it may not compute changed lines correctly.

In any case, the previous paragraph is not really important for fixing Pull Request analysis. You need to provide the following parameters:

  • sonar.pullrequest.branch - the name of the branch in this pull request
  • sonar.pullrequest.base - the name of the branch you want to merge into
  • sonar.pullrequest.key - the pull request number, for example 1234
  • sonar.pullrequest.provider=github
  • sonar.pullrequest.github.repository - the “slug” of your repository on GitHub, for example orgname/reponame

And you need to make sure the target branch (the value of sonar.pullrequest.base) is fetched.


(Vitaly Karasik) #13

Now I’m testing from Jenkins CI, using “-Dsonar.branch.name=$GIT_BRANCH” for branch analyze and “-Dsonar.pullrequest.branch=$GIT_BRANCH -Dsonar.pullrequest.key=$CHANGE_ID” for PR analyze.
In my sonar.gradle I have property “sonar.projectKey” and “sonar.organization”.
I’m testing PR analyze now. I do see changes line on my code, and SonarCloud portal shows proper link to my GitHub PR and “for merge with master”.
I added “-Dsonar.pullrequest.provider=github -Dsonar.pullrequest.base=master” and “sonar.pullrequest.github.repository”, it didn’t improved situation - on my GitHub PR page I do see Jenkins status, but no SonarCloud one.


(Janos Gyerik) #14

As I mentioned already, don’t use -Dsonar.branch.name in a Pull Request analysis. It’s pointless and potentially confusing.

I suggest to first try to get PR analysis working correctly locally, not from Jenkins. Once you manage to get that working, then you should be able to port that to Jenkins. Without knowing the actual values of $GIT_BRANCH and $CHANGE_ID, and without knowing the specific PR in question on GitHub, I have no way to verify that your setup is correct. Is the project publicly visible? If yes, you could share it, and the exact command that you are running. Then I may be able to pinpoint the problem.


(Vitaly Karasik) #15

Already got it - as I wrote I use

My real command is
./gradlew sonarqube -Dsonar.pullrequest.branch=PR-288 -Dsonar.pullrequest.key=288 -Dsonar.pullrequest.provider=github -Dsonar.pullrequest.base=master -Dsonar.pullrequest.github.repository=Company/project

Is there a chance that I should configure some GitHub hook manually? As I wrote, in SonarCloud portal I do see right URL of my PR.

Unfortunately, no. I’ll be glad to share credentials in private. Or - I’ll try to reproduce the issue with some opensource project fork.


(Janos Gyerik) #16

Could you clarify what is missing at this point? Also:

  • On the SonarCloud page of your project, do you see warnings?
  • On the Conversation tab of the pull request on GitHub, do you see the status of the SonarCloud analysis?
  • On the Checks tab of the pull request on GitHub, do you see a check for SonarCloud analysis?

(Vitaly Karasik) #17

I don’t see SonarCloud check on conversation page - see attached screenshot.

Yes, I see updated code for my PR (no warnings because it’s just a PR for Sonar tests)

I see it as “Queued” - see attached screenshot.
01

Thank you again, Vitaly


(Vitaly Karasik) #18

This seems me like an issue with Sonar GitHub app communication to GitHub, but I see no way to debug it.


(Vitaly Karasik) #19

I’ll appreciate if someone from SonarSource team will be able to assist.
I already read all related documentation, plus similar threads: SonarCloud now not updating GitHub PR and Checks , https://community.sonarsource.com/t/github-pull-request-decoration-doesnt-work-if-provider-or-token-as-command-line-properties/1411;
but it didn’t help.

PS:
IMHO:
While SonarSource core is very strong and useful product, its integration is confusing. I understand that there are many plugins for different build tools (Gradle, Maven, …), CIs (Travis, Jenkins, …) and version-control systems. While there are blog posts and forum threads, I see no full “User manual”, updated for current state of SonarSource ecosystem.


(Vitaly Karasik) #20

SOLVED by replacing GitHub Jenkins plugin “branch discovery strategy” - thanks to PR decoration for GitHub no longer working.

“I’m not sure if it could be a problem, but how our jenkins build works is it merges master into the branch and creates a new branch named PR- eg PR-123. I think this is standard jenkins behaviour though, and we don’t build the actual branch during pull request as we don’t care about it. But I guess github wont have the same sha1?”