Sonar Plugin for Jenkins crashes the Blue Ocean view

This was working before the weekend and now it somehow prevent us from seeing our PR builds which are tied to SonarCloud. From our Jenkins GUI the pages get stuck and eventually time out.

This will significantly impact our business since our developers are no longer able to look at the pipeline stages failures. Other pipelines that do not include Sonar are fine.

We are investigating to check if something has changed on our side, but the deeper problem here is that if the Sonar Plugin gets a 404 back from sonarcloud.io it should not disable the entire Blue Ocean view for the job.

Oct 07, 2019 4:36:26 PM WARNING hudson.plugins.sonar.client.SQProjectResolver resolve
Error fetching project information
org.sonarqube.ws.client.HttpException: Error 404 on https://sonarcloud.io/api/ce/task?id=AWxvGMnohvfIjRvd_pyK : {"errors":[{"msg":"No activity found for task 'AWxvGMnohvfIjRvd_pyK'"}]}
	at org.sonarqube.ws.client.BaseResponse.failIfNotSuccessful(BaseResponse.java:36)
	at hudson.plugins.sonar.client.HttpClient.getHttp(HttpClient.java:38)
	at hudson.plugins.sonar.client.WsClient.getCETask(WsClient.java:51)
	at hudson.plugins.sonar.client.SQProjectResolver.requestCETaskDetails(SQProjectResolver.java:83)
	at hudson.plugins.sonar.client.SQProjectResolver.resolve(SQProjectResolver.java:67)
	at hudson.plugins.sonar.action.SonarCacheAction.get(SonarCacheAction.java:77)
	at hudson.plugins.sonar.action.SonarCacheAction.get(SonarCacheAction.java:52)
	at hudson.plugins.sonar.action.SonarProjectActionFactory.createProjectPage(SonarProjectActionFactory.java:118)
	at hudson.plugins.sonar.action.SonarProjectActionFactory.createFor(SonarProjectActionFactory.java:83)
	at hudson.plugins.sonar.action.SonarProjectActionFactory.createFor(SonarProjectActionFactory.java:42)
	at hudson.model.Actionable.createFor(Actionable.java:114)
	at hudson.model.Actionable.getAction(Actionable.java:337)
	at io.jenkins.blueocean.rest.impl.pipeline.PrimaryBranch.lambda$resolve$0(PrimaryBranch.java:23)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:649)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at com.google.common.collect.Iterators.find(Iterators.java:738)
	at com.google.common.collect.Iterables.find(Iterables.java:656)
	at io.jenkins.blueocean.rest.impl.pipeline.PrimaryBranch.resolve(PrimaryBranch.java:22)
	at io.jenkins.blueocean.rest.impl.pipeline.MultiBranchPipelineImpl.getWeatherScore(MultiBranchPipelineImpl.java:172)
	at io.jenkins.blueocean.commons.stapler.export.MethodProperty.getValue(MethodProperty.java:72)
	at io.jenkins.blueocean.commons.stapler.export.ExportInterceptor$1.getValue(ExportInterceptor.java:46)
	at io.jenkins.blueocean.commons.stapler.Export$BlueOceanExportInterceptor.getValue(Export.java:196)
	at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:136)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:224)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:224)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:199)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:219)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:183)
	at io.jenkins.blueocean.commons.stapler.Export.toJson(Export.java:90)
	at io.jenkins.blueocean.commons.stapler.Export.toJson(Export.java:59)
	at io.jenkins.blueocean.commons.stapler.Export.toJson(Export.java:46)
	at io.jenkins.blueocean.preload.PipelineStatePreloader.getFetchData(PipelineStatePreloader.java:66)
	at io.jenkins.blueocean.preload.RESTFetchPreloader.getStateJson(RESTFetchPreloader.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
	at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
	at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
	at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
	at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
	at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
	at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
	at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
	at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
	at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
	at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
	at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
	at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
	at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:147)
	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
	at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
	at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
	at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
	at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
	at org.kohsuke.stapler.jelly.ScriptInvoker.execute(ScriptInvoker.java:56)
	at org.kohsuke.stapler.jelly.ScriptInvoker.execute(ScriptInvoker.java:43)
	at org.kohsuke.stapler.Facet.handleIndexRequest(Facet.java:282)
	at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:99)
	at org.kohsuke.stapler.IndexViewDispatcher.dispatch(IndexViewDispatcher.java:32)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:717)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:676)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:873)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:246)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
	at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:88)
	at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:114)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:92)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
	at java.lang.Thread.run(Thread.java:748)

Things seems to have gotten back to normal, although that 404 url is still returning the same json but we assume that the Sonar Plugin is no longer trying to access that url.

This is definitely a bug in the Jenkins Sonar Plugin since making query to the SonarCloud server should not cause an outage of the Jenkins GUI.

The issue is back. Is there an escalation channel for Sonar issue that impact a SonarCloud paying customer?

Hi Stéphane,

The task referenced in your logs does not exist on SonarCloud. For the time being I see three possible reasons to that:

  • the base URL configured in Jenkins has been changed during the analysis
  • the base URL is configured in both Jenkins and project sources (see parameter sonar.host.url) and they have different values.
  • the project has been dropped from SonarCloud during the analysis. Could you please share privately the project key please?

Does that make sense?

For the side-effect on Blue Ocean, could you please share your version of Jenkins/Blue Ocean?

Hi Simon,

We have the sonar host url configured through the plugin on the Jenkins master configuration:

To run the analysis we use the official sonarsource/sonarcloud-scan docker image (so it is always the one tagged latest, with an always pull policy) and call a shell script step directly on it. Snippet from our Jenkinsfile:

            container('sonar-scanner') {
                withSonarQubeEnv('SonarCloud') {
                    sh 'sonar-scanner.sh'
                }
            }

Our sonar-project.properties contains sonar.projectKey and sonar.organization but not sonar.host.url. The shell script does not set sonar.host.url either. We let the Sonar Plugin take care of that which is what the withSonarQubeEnv('SonarCloud') is for in our Jenkinsfile.

The project has been created a few months ago and we have not deleted it since. I’ll send you that information privately.

For the versions:

  • Jenkins 2.190.1
  • Blue Ocean 1.19.0
  • SonarQube Scanner for Jenkins 2.9

I’m trying to find out how to send private messages so I can share the project key…

Investigating the reason of the 404 response. Meanwhile the Blue Ocean page should be resilient to this kind of error. Bug is tracked in https://jira.sonarsource.com/browse/SONARJNKNS-309.

Update for others:

  • Simon PM’ed me for the private details. Only admins can start PMs for this forum.
  • We disabled the Sonar Plugin on Jenkins and use regular Jenkins secrets and environment variables to configure the sonar runner. This shields us from the bug so that BlueOcean can work normally and there is no visible loss of functionality for our developers.
2 Likes

We have a similar problem.

It looks like Blue Ocean is trying to get meta information for all projects when loading a view. It creates a huge number of requests to the SonarQube server.

Some of them receive a 404 response due to the fact that these scans have already been deleted on the sonar server.

For our installation, the time until Jenkins receives the necessary information and gives it to the Blue Ocean can be 180 seconds.
Such a long response sometimes leads to 504.

Blue Ocean doesn’t have a function to display links to SonarQube scan results, that is why Blue Ocean is loading this data in vain.

We haven’t found a way to disable sonar widget or fix the issue.
It is not clear who can fix this problem, Blue Ocean team or SonarQube team.

Jenkins: v2.303.1
BlueOcean: v1.24.8
SonarQube Scanner for Jenkins: v2.13.1
Sonar Server: v8.9



1 Like

We have found a solution for our problem. BlueOcean requests information about favorite jobs.

And because of the plugin Autofavorite for Blue Ocean, their number is constantly growing.

To speed up the loading of the BlueOcean view, you should to disable autofavoriteEnabled in the user config file.
replace <autofavoriteEnabled>true</autofavoriteEnabled> to <autofavoriteEnabled>false</autofavoriteEnabled> in "jenkins_home/users/*/config.xml

And remove unnecessary jobs from favorites. It could be done in the same user config file, replace <boolean>true</boolean> to <boolean>false</boolean> in “jenkins_home/users/*/config.xml”

1 Like

It’s indeed the user favoriting that’s messing things up, combined with the Sonar Plugin. I ran this in Jenkins’s script console to fix things:

User.getAll().each { user ->
  if (user.id == "SYSTEM")
      return

  user.getProperty(hudson.plugins.favorite.user.FavoriteUserProperty.class).data.clear()
  user.getProperty(io.jenkins.blueocean.autofavorite.user.FavoritingUserProperty.class).setAutofavoriteEnabled(false)
  user.save()
}

This disables auto-favoriting for all users and removes all favorites. My Blue Ocean page load time went from 90 seconds to 61 ms.
Note that there is something clearly broken with the plugin as opening up jobs with analysis that doesn’t exist anymore is still taking a long time. We’ll disable the plugin and just pass the correct parameters manually.

2 Likes

Man, Big thank you!
We suspected sonarqube contributing to long load of BO (if I put 127.0.0.1 sonar-server to hosts it works also fast) but this solution improved load speed a lot. Thank you!