versions
SonarQube Enterprise Edition Version 7.9.6 (build 41879)
error observed
Jenkins is searching all branches analysis using OkHttp thread and in case the analysis was canceled on Sonarqube, the thread remain into the pool for 5 minutes. Repeating this we reach the 500 limit threads of Jenkins after the Jenkins cannot initialize new thread
steps to reproduce
Jenkins process a feature branch and do an analysis with sonarqube.
Analysis is OK and published in sonarqube
After a while (Key: sonar.dbcleaner.daysBeforeDeletingInactiveShortLivingBranches) the analysis is canceled.
The web page of the organization and project in Jenkins is searching all informationās about each branches, including the old branches where the analysis has been deleted.
The OkHttp thread used remain active for 5 minutes.
Repeating the reload of the web page increase the number of Okhttp thread and reaches 500 threads
At that point Jenkins is not able to initialize new thread and is stuck
We found this problem when many people was loading the web page of the projects during a consolidation release. People was checking the progress of the pipeline in jenkins and suddenly the Jenkins was crashing.
As remediation in short term, we have reduced the number of old project by filtering it from Jenkins, but we cannot guarantee a stable situation for ever.
Is the issue related to the Jenkins plugin āSonarScanner for Jenkinsā? If so, whatās the version of Jenkins and the SonarScanner for Jenkins?
Do you have any error stack trace that you could show us? Also perhaps your pipeline configuration?
Whatās the role of the key sonar.dbcleaner.daysBeforeDeletingInactiveShortLivingBranches in this problem? Is it that when a branch gets deleted, thereās a connection from Jenkins that hangs? However at the beginning you mention that itās due to a canceled analysis?
Hi Duarte,
I understand, we are trying several month to solve the issueā¦
I donāt think the problem is related to the plugin āSonarScanner for Jenkinsā, because the problem happen without a running analysis. What I donāt know, if Jenkins is using the plugin to get information from Sonarqube.
Itās a GitHub Organization pipeline, it scans a GitHub organization for all repositories matching some defined markers. As soon you enters on the Organization (https://jenkins-server/jenkins/job/OrganizationName/) you have the list of organizationās project. When you select a project (https://jenkins-server/jenkins/job/OrganizationName/job/ProjectName/) you have the branch list. At that point, in case some branches exist in Jenkins and the analysis in Sonarqube have been cancelled, we found additional OkHttp thread on the Jenkins server. The thread remain for 5 minutes and after disappears. If different people enter on that web page on the same time (<5min), the thread count increase and if reaches 500 the Jenkins server is stuck.
Unfortunately I didnāt found any error stack trace
The role of the key is that we have the problem when the branch in Jenkins is present and the analysis in sonarqube has been cancelled. The key define the time before delete the analysis.
I think itās a communication problem between Jenkins and Sonarqube, I suppose Jenkins ask some details about the analysis, Sonarqube answer with an error or an exception, but Jenkins is not handling correctly the error or the exception, and didnāt release the thread.