Facing issues in sonar webhook after Upgrade to enterprise edition 8.1 from developer edition 7.1

Hi Team,
I am facing issues with Sonarqube webhooks, I have upgraded my sonar from 7.1 developer edition to 8.1 enterprise edition. After that my scanning is working fine, but when I tried to execute the quality gate from Jenkins(We are using Jenkins pipeline) it is not working.
My Jenkins job stuck with below logs
Checking status of SonarQube task XXX on server ‘testsonar’
SonarQube task ‘XXXXX’ status is 'IN_PROGRESS’
I have already set up my Jenkins URL in sonarqube webhooks like below
https://username :password@myjenkinsurl/sonarqube-webhook/
It is not working.

Hi,

Does SonarQube show that the webhook was called? Does the call show up in your Jenkins log?

 
Ann

Hi Ann,

Thanks for the response.

I and Raj in the same team and working together on this issue.
Here is Jenkins Logs when it stuck at waitForQualityGate stage, it exit on timeout or stuck at the same stage until I abort the job.

[Pipeline] waitForQualityGate
Checking status of SonarQube task '***********' on server 'sonar1'
SonarQube task '*******' status is 'IN_PROGRESS'

When quality gate passes the logs are following.

[Pipeline] waitForQualityGate **03:26:15** Checking status of SonarQube task '****************' on server 'sonar1' **03:26:15** SonarQube task '****************' status is 'SUCCESS' **03:26:15** SonarQube task '****************' completed. Quality gate is 'OK' [Pipeline] echo **03:26:15** SonarQube analysis successful

Can you suggest what can be an issue?

Here are our history and Jenkins setup details.
We have developer edition and we are planning to upgrade the enterprise edition
We purchased it and installed for the test machine to make sure everything is working fine or not.

Test setup

  1. We are using a docker image on kops cluster
  2. In Jenkins, we have 2 sonarqube server setup, 1: Developer Edition, 2: Enterprise Edition
  3. For our test, we are using 2nd sonar env in Jenkins pipeline with withSonarQubeEnv(’’) method

Can you please help us on this thread because this is stopping us to upgrade and our existing license is already reached to limit, which threaten us to make CI jobs failing.

Hi,

The questions are the same. :slight_smile:

 
Ann

Hi @ganncamp,

from sonar I can see the last delivery option display the Never for my all webhooks. Attached the screenshot.
I have tested my Jenkins connectivity from sonar container using curl and telnet and it is working as expected.

Hi Ann,

We added the following block which makes the quality gate success sometimes and fails sometimes. We won’t be preferring this as it is a workaround not the perfect solution.

stage ('Quality Gate') {
                when { expression { return pipelineParams.QualityGate } }
                steps {
                    script {
                        def maxRetry = 2
                        def i = 0
                        for (i=0; i<maxRetry; i++){
                            try {
                                timeout(time: 5, unit: 'SECONDS') {
                                    def qualitygate = waitForQualityGate()
                                    if (qualitygate.status != "OK") {
                                        echo "Pipeline aborted due to quality gate coverage failure: ${qualitygate.status}"
                                    }
                                    else {
                                        echo "SonarQube analysis succesfull"
                                    }
                                }
                            }
                            catch(Exception e) {
                                if (i == maxRetry-1) {
                                    println "No of Tries : ${i}, ${e}"
                                    throw e        
                                }
                            }
                        }                            
                    }
                }
            }

I am not sure what logs will helpful to you so I am attaching access.log and ce.log for both the scenarios success and failure.

failed-qualitygates.txt (14.5 KB) success-qualitygate.txt (15.6 KB)

Hi,

Thanks for the screenshots. It’s odd to me that your webhooks have never been delivered. Are you sure your background task processing completes successfully? Can I have a screenshot of that? E.G.

 
Ann

Hi Ann,

Please check the following screen of background tasks, all say completed.

Thanks,
Chandani

Hi Chandani,

Thanks for the screenshot. That’s the global Background tasks page. Was the Webhooks screenshot from the global level or project-specific?

 
Ann

Hi Ann,

Webhook screenshot was global level settings.
Background tasks screenshot include all tasks no filter

Chandani

Hi, I am coming here because I got the same exact issue after upgrade.
We have the SonarQube 8.1 installed in an AWS Fargate container and the Jenkins build halts at

SonarQube task ‘XXXXX’ status is 'IN_PROGRESS’

When we enabled trace I see the below logs for the ID

2020.02.12 16:25:52 TRACE ce[XXXXXX_X-XXXXX][sql] time=1ms | sql=delete from ce_task_input where task_uuid in ( ? ) | params=XXXXXX_X-XXXXX
2020.02.12 16:25:52 TRACE ce[XXXXXX_X-XXXXX][sql] time=1ms | sql=delete from ce_queue where uuid=? and status = ? | params=XXXXXX_X-XXXXX, IN_PROGRESS
2020.02.12 16:25:52 INFO ce[XXXXXX_X-XXXXX][o.s.c.t.CeWorkerImpl] Executed task | project=service.name | type=REPORT | pullRequest=branch/name | id=XXXXXX_X-XXXXX | submitter=admin | status=SUCCESS | time=1362ms

This works fine for 3 or 4 days after I restart the container

But this is recurring (4th time now) and we are still clueless.

Steps we made for checking this further

  1. made sure that the CPU/memory usage of the container is below threshold
  2. tested with multiple pipeline and all getting failed during the issue
  3. checked the logs in the DB (AWS RDS) and there were nothing significant

Background task for the specific ID is showing success as well

Any help is appreciated
Thanks

Between a sleep of 20 seconds (10 sec didnt work out) between the stages worked though its not a proper solution in this case

stage(“Quality Gate”) {
steps {
sleep(20)
timeout(time: 1, unit: ‘HOURS’) {
waitForQualityGate abortPipeline: true
}
}
}

Thanks for the ce.log, your issues is coming from :

2020.02.06 17:40:39 INFO ce[AXAbluCuHBdiwf_jyzdX][c.k.s.s.c.c.AbstractSlackNotifyingComponent] Refreshing settings
2020.02.06 17:40:39 INFO ce[AXAbluCuHBdiwf_jyzdX][c.k.s.s.c.c.AbstractSlackNotifyingComponent] Refreshing project configs
2020.02.06 17:40:39 INFO ce[AXAbluCuHBdiwf_jyzdX][c.k.s.s.c.c.AbstractSlackNotifyingComponent] SlackNotifierProp.CONFIG=[1]
2020.02.06 17:40:39 INFO ce[AXAbluCuHBdiwf_jyzdX][c.k.s.s.c.c.AbstractSlackNotifyingComponent] Found project configuration [ProjectConfig{projectKey=‘hms*’, slackChannel=‘sonarqube-analysis’, qgFailOnly=false}]
2020.02.06 17:40:39 INFO ce[AXAbluCuHBdiwf_jyzdX][c.k.s.s.e.t.SlackPostProjectAnalysisTask] Analysis ScannerContext: [{}]
2020.02.06 17:40:39 INFO ce[AXAbluCuHBdiwf_jyzdX][c.k.s.s.c.c.AbstractSlackNotifyingComponent] Could not find config for project [CI-backend-***********] in [{hms=ProjectConfig{projectKey='hms’, slackChannel=‘sonarqube-analysis’, qgFailOnly=false}}]
2020.02.06 17:40:39 ERROR ce[AXAbluCuHBdiwf_jyzdX][o.s.c.t.s.ComputationStepExecutor] Execution of listener failed
java.lang.AbstractMethodError: Receiver class com.koant.sonar.slacknotifier.extension.task.SlackPostProjectAnalysisTask does not define or inherit an implementation of the resolved method ‘abstract java.lang.String getDescription()’ of interface org.sonar.api.ce.posttask.PostProjectAnalysisTask.
at org.sonar.ce.task.projectanalysis.api.posttask.PostProjectAnalysisTasksExecutor.executeTask(PostProjectAnalysisTasksExecutor.java:124)
at org.sonar.ce.task.projectanalysis.api.posttask.PostProjectAnalysisTasksExecutor.finished(PostProjectAnalysisTasksExecutor.java:109)
at org.sonar.ce.task.step.ComputationStepExecutor.executeListener(ComputationStepExecutor.java:91)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:63)
at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:209)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:191)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:158)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:133)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:85)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)

It seems you have a plugin that contains a class “SlackPostProjectAnalysisTask” implementing the org.sonar.api.ce.posttask.PostProjectAnalysisTask API, which makes post tasks failing, including web hooks.

You should remove this plugin, then try again.

Are you using docker or server deployment?