[Jenkins] Don't fail build when SonarQube analysis is broken (db down, etc)


(Jeff Certain) #1

SonarQube (server 6.7.5)

We’re in the process of rolling out SQ integration throughout the org. However, this isn’t always smooth – our sonarqube git user got accidentally suspended the other day, we’ve had db issues, etc.

Rather than break all builds when the SQ PR builder integration is down, I’d like to not break the jenkins build if the SQ scanner build step fails. However, I didn’t find any way to set this up in Jenkins.

Is there a way to set the SonarQube Scanner plugin to not fail in the case where it can’t push the analysis to the SQ server?


(G Ann Campbell) #2

Hi,

We’ve put no effort into this topic. We’d rather make sure the SQ server stays up and available. If we can help you on that topic, please feel free to open new threads with the pieces of that.

 
Ann


(Jeff Certain) #3

Hi Ann,

I agree – the server should be up and available. However, particularly for PR builds, there’s a but of an ecosystem around that.

Here’s where I’m coming from with my request: we’ve recently had a couple of incidents in which that ecosystem was broken – once by a configuration issue on the SQ server and once as a result of a permissions issue on the github side of things. In both of these cases, the team was heavily impacted; in keeping with Murphy’s law, we had releases in flight with patches that needed PR.

Yes, I got people moving to fix the issue through both official and back channels, and the issue was resolved moderately quickly. Still, it was painful for all involved, and a more graceful way to fail would be appreciated.

What I’m trying to avoid is a single point of failure in a large organization. We’re also in the process of rolling out SQ to several thousand developers, and I’d rather not have SQ perceived as being burdensome or unusable.

~ Jeff


(G Ann Campbell) #4

Hi Jeff,

Thanks for sharing your viewpoint. I certainly (probably predictably) share your concerns about the perception of SonarQube. After brief internal discussions, the word “pipeline” was thrown out, but I’ll be frank that I don’t know enough on that subject to actually know if it would help or not.

At the same time, I have two thoughts to share:

  1. This ought to be only a concern while you work out the initial kinks. Once it’s fully up and running, our experience is that it’s stable.
  2. I understand that Murphy’s law made this incident particularly painful, but analysis is a safety step in your process. The analogy is a bit strong, but would you really want your pilot to go forward with takeoff if he hadn’t been able to run the engine checks?

 
:-/
Ann


(Jeff Certain) #5

I agree. In an ideal world, this sort of issue would be infrequent/non-existent once the system was stable. Unfortunately, cows aren’t round.

To respond to your analogy in kind, I would be willing to undertake the risk if the pilot needed to get a carefully maintained aircraft off the runway when there were enemy bombers inbound. As you say, this should be a rare occurrence, and in normal day-to-day operations it’s not a problem to wait for any SQ issues to be resolved. However, if there are P0/P1 production issues that are blocked because SQ isn’t available… well, I might be willing to take the risk, particularly since, over time, the team should be improving (and possibly using sonarlint to catch issues before we ever get to full SQ analysis).

~ Jeff


(G Ann Campbell) #6

Touche, and thanks for the laugh. :smile:

After a small amount of research it does look like pipelines are your answer; they apparently support try / catch so I guess you’d try to run analysis and then catch and interpret any failures. The challenge will be distinguishing between an error caused by “analysis fell over dead” and “Quality Gate status == fail”… assuming you actually want to do the latter. If you don’t, it gets really simple.

 
Ann