SonarQube Scanner Testing

Curious about how people are currently testing SonarQube’s results following updates and/or patches. I was thinking about how to automate this. We use jenkins for builds so if you had a set of code with known issues you could use jenkins pipelines to run the sample code, check the output, and then if the results from the sample code don’t match, break the build. If they do match proceed.

Curious if anyone else has done something like this or if it would be an interesting feature for either SQ server or as a new feature for scanners/plugins to have.

jeff

Hi jeff,

You’re aware that with each new version, at least some of SonarQube’s built in profiles will have been updated and rules gotten smarter, right? So it’s quite likely that you’ll have a break after every upgrade, depending on the languages under analysis.

Is the goal to detect when rules get smarter (meaning both fewer false positives and fewer false negatives)? Or is this about detecting when they get dumber?

And in case it helps, we have a dedicated internal server where we do this sort of testing extensively.

 
Ann

Yep, definitely aware of these subtle changes. I was more looking for a way to check for major breakages after an upgrade. More of a smoke test. When we apply an update now we don’t really know if plugin XYZ is no longer working or if some feature on the server side is not working correctly.

1 Like