Sonar Plugin for Jenkins crashes the Blue Ocean view

sodul · October 7, 2019, 4:36pm

This was working before the weekend and now it somehow prevent us from seeing our PR builds which are tied to SonarCloud. From our Jenkins GUI the pages get stuck and eventually time out.

This will significantly impact our business since our developers are no longer able to look at the pipeline stages failures. Other pipelines that do not include Sonar are fine.

We are investigating to check if something has changed on our side, but the deeper problem here is that if the Sonar Plugin gets a 404 back from sonarcloud.io it should not disable the entire Blue Ocean view for the job.

Oct 07, 2019 4:36:26 PM WARNING hudson.plugins.sonar.client.SQProjectResolver resolve
Error fetching project information
org.sonarqube.ws.client.HttpException: Error 404 on https://sonarcloud.io/api/ce/task?id=AWxvGMnohvfIjRvd_pyK : {"errors":[{"msg":"No activity found for task 'AWxvGMnohvfIjRvd_pyK'"}]}
	at org.sonarqube.ws.client.BaseResponse.failIfNotSuccessful(BaseResponse.java:36)
	at hudson.plugins.sonar.client.HttpClient.getHttp(HttpClient.java:38)
	at hudson.plugins.sonar.client.WsClient.getCETask(WsClient.java:51)
	at hudson.plugins.sonar.client.SQProjectResolver.requestCETaskDetails(SQProjectResolver.java:83)
	at hudson.plugins.sonar.client.SQProjectResolver.resolve(SQProjectResolver.java:67)
	at hudson.plugins.sonar.action.SonarCacheAction.get(SonarCacheAction.java:77)
	at hudson.plugins.sonar.action.SonarCacheAction.get(SonarCacheAction.java:52)
	at hudson.plugins.sonar.action.SonarProjectActionFactory.createProjectPage(SonarProjectActionFactory.java:118)
	at hudson.plugins.sonar.action.SonarProjectActionFactory.createFor(SonarProjectActionFactory.java:83)
	at hudson.plugins.sonar.action.SonarProjectActionFactory.createFor(SonarProjectActionFactory.java:42)
	at hudson.model.Actionable.createFor(Actionable.java:114)
	at hudson.model.Actionable.getAction(Actionable.java:337)
	at io.jenkins.blueocean.rest.impl.pipeline.PrimaryBranch.lambda$resolve$0(PrimaryBranch.java:23)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:649)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at com.google.common.collect.Iterators.find(Iterators.java:738)
	at com.google.common.collect.Iterables.find(Iterables.java:656)
	at io.jenkins.blueocean.rest.impl.pipeline.PrimaryBranch.resolve(PrimaryBranch.java:22)
	at io.jenkins.blueocean.rest.impl.pipeline.MultiBranchPipelineImpl.getWeatherScore(MultiBranchPipelineImpl.java:172)
	at io.jenkins.blueocean.commons.stapler.export.MethodProperty.getValue(MethodProperty.java:72)
	at io.jenkins.blueocean.commons.stapler.export.ExportInterceptor$1.getValue(ExportInterceptor.java:46)
	at io.jenkins.blueocean.commons.stapler.Export$BlueOceanExportInterceptor.getValue(Export.java:196)
	at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:136)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:224)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:224)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:199)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:219)
	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:183)
	at io.jenkins.blueocean.commons.stapler.Export.toJson(Export.java:90)
	at io.jenkins.blueocean.commons.stapler.Export.toJson(Export.java:59)
	at io.jenkins.blueocean.commons.stapler.Export.toJson(Export.java:46)
	at io.jenkins.blueocean.preload.PipelineStatePreloader.getFetchData(PipelineStatePreloader.java:66)
	at io.jenkins.blueocean.preload.RESTFetchPreloader.getStateJson(RESTFetchPreloader.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
	at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
	at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
	at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
	at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
	at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
	at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
	at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
	at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
	at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
	at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
	at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
	at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
	at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:147)
	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
	at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
	at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
	at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
	at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
	at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
	at org.kohsuke.stapler.jelly.ScriptInvoker.execute(ScriptInvoker.java:56)
	at org.kohsuke.stapler.jelly.ScriptInvoker.execute(ScriptInvoker.java:43)
	at org.kohsuke.stapler.Facet.handleIndexRequest(Facet.java:282)
	at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:99)
	at org.kohsuke.stapler.IndexViewDispatcher.dispatch(IndexViewDispatcher.java:32)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:717)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:676)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:873)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:246)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
	at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:88)
	at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:114)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:92)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
	at java.lang.Thread.run(Thread.java:748)

sodul · October 7, 2019, 8:20pm

Things seems to have gotten back to normal, although that 404 url is still returning the same json but we assume that the Sonar Plugin is no longer trying to access that url.

This is definitely a bug in the Jenkins Sonar Plugin since making query to the SonarCloud server should not cause an outage of the Jenkins GUI.

sodul · October 8, 2019, 9:04pm

The issue is back. Is there an escalation channel for Sonar issue that impact a SonarCloud paying customer?

simon.brandhof · October 9, 2019, 9:02am

Hi Stéphane,

The task referenced in your logs does not exist on SonarCloud. For the time being I see three possible reasons to that:

the base URL configured in Jenkins has been changed during the analysis
the base URL is configured in both Jenkins and project sources (see parameter sonar.host.url) and they have different values.
the project has been dropped from SonarCloud during the analysis. Could you please share privately the project key please?

Does that make sense?

simon.brandhof · October 9, 2019, 8:59am

For the side-effect on Blue Ocean, could you please share your version of Jenkins/Blue Ocean?

sodul · October 9, 2019, 3:17pm

Hi Simon,

We have the sonar host url configured through the plugin on the Jenkins master configuration:

To run the analysis we use the official sonarsource/sonarcloud-scan docker image (so it is always the one tagged latest, with an always pull policy) and call a shell script step directly on it. Snippet from our Jenkinsfile:

            container('sonar-scanner') {
                withSonarQubeEnv('SonarCloud') {
                    sh 'sonar-scanner.sh'
                }
            }

Our sonar-project.properties contains sonar.projectKey and sonar.organization but not sonar.host.url. The shell script does not set sonar.host.url either. We let the Sonar Plugin take care of that which is what the withSonarQubeEnv('SonarCloud') is for in our Jenkinsfile.

The project has been created a few months ago and we have not deleted it since. I’ll send you that information privately.

For the versions:

Jenkins 2.190.1
Blue Ocean 1.19.0
SonarQube Scanner for Jenkins 2.9

sodul · October 9, 2019, 3:23pm

I’m trying to find out how to send private messages so I can share the project key…

simon.brandhof · October 11, 2019, 10:03am

Investigating the reason of the 404 response. Meanwhile the Blue Ocean page should be resilient to this kind of error. Bug is tracked in https://jira.sonarsource.com/browse/SONARJNKNS-309.

sodul · October 30, 2019, 6:52pm

Update for others:

Simon PM’ed me for the private details. Only admins can start PMs for this forum.
We disabled the Sonar Plugin on Jenkins and use regular Jenkins secrets and environment variables to configure the sonar runner. This shields us from the bug so that BlueOcean can work normally and there is no visible loss of functionality for our developers.