Thoughts on driving cross-platform C++ builds with varying code coverage states

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    • SonarQube Developer Edition Version 9.4 (build 54424)
    • SonarScanner 4.7.0.2747
  • what are you trying to achieve
    • Parallel builds across multiple platforms, some of which can contribute to code coverage analysis (desktop; Windows, MacOS, Linux), and some which cannot (mobile and game console).
  • what have you tried so far to achieve this
    • Investigation into methodologies to route individual builds to different quality gates.
    • Investigation into separating contributing and non-contributing code coverage build permutations into separate Sonar projects.

Breakdown
I am building C++ projects that support many different hardware/OS platforms, some of which are capable of contributing code coverage reports via Bullseye, gcov, etc. and some which cannot (typically this is because tests are run remotely on platform specific hardware like game console developer kits, mobile devices, etc.). In order to keep these parallel builds from clobbering or obfuscating quality history, they are bucketed into different “branches” in the project using sonar.branch.name. However, this is running afoul of code coverage rules in the quality gate.

Simply put, the build permutations that don’t contribute code coverage will fail the quality gate if it contains code coverage rules. Makes sense. However, simply removing the rules removes their enforcement on the build permutations that do contribute coverage. Again, simple enough to understand.

My question, then, is there a feasible strategy for solving this that doesn’t involve multiple projects? That is not a reasonable solution for us, as it will impact reporting and also artificially inflate line counts for licensing purposes.

The ideal solution would be to set which quality gate a build runs, allowing build permutations with no code coverage to select a gate that does not require it as a condition. This feature was once available using the sonar.qualitygate parameter, but this has not been available in Sonarqube for quite some time. Using the server’s api/qualitygates/select web API is also impractical, since builds are run in parallel and sonar scans would need to be serialized across build permutations in order to guarantee gate state for each permutation.

I have some ideas on how I might go about resolving this, but thought I’d pick the community’s brain and see what others think. :slight_smile:

2 Likes

Hello Eli, I am not a technical person, and most likely my answer is wrong or incomplete, but have you considered that for each project you can specify a Quality gate and with Developer Edition you can create “Applications” Applications | SonarQube Docs which allow you to combine 2 or more projects? let’s say even if you split them in half, this would not increase your line count, anyway just a thought, from another Eli

1 Like

Hi Eli,

Thanks for the reply!

Applications would be an ideal approach for a lot of the workarounds I’m having to use for multiple cross-platform build permutations in C++ as well as solving this particular problem, except that it incurs the aforementioned problem with duplicating LoC across projects. This is because the vast majority of the code in my projects is shared across platforms, and therefore must be considered in the contributing source to any project defined in this manner.

After a bit of experimentation, I’ll also add that Applications do not appear to support tracking pull requests across their associated Projects. This is fairly sensible, as there is no guarantee that an SCM’s PR notation system can be reliably matched in this manner (e.g. does a git PR-123 on project A really match up with a similarly-named PR-123 on project B?).

This might be an opportunity to introduce a configuration option to Applications that states something to the effect of “all of the projects within this Application are from the same source”, which could enable PR tracking across them as well as not duplicating line counts for matching files. That would definitively solve my problem as well as obviate the need for many of the workarounds I’ve had to do to support multiple build permutations (the aforementioned branch name hack, for example).

Hi,

This seems related to this topic

And specifically to this post:

 
HTH,
Ann

1 Like

Definitely germane to this conversation, thanks for the heads-up!

1 Like

The ideal solution to this problem is definitely enabling support for multiple permutations of a build that contribute to, and should be aggregated into, a single reporting outcome on the server. Some things to keep in mind:

  • There are a wide variety of factors that can contribute to the permutations of a C++ build: OS type, OS version, compiler version, STL implementation, disposition of various compiler/linker flags, etc.; too many to try and create a static mapping for. This solution should be flexible, e.g. surfacing permutations based on a user-set parameter such as sonar.buildString.
  • Permutations will likely contribute unequally to satisfying the rules of a quality gate; some additional flexibility in rules creation will likely be needed to allow a project to consider either individual permutations or their aggregated results. The former may prove challenging given what constitutes a rule pass might vary widely across permutations.
2 Likes

Is there any progress or recent work on the multi-platform analysis?

Hi,

We’re actively working on this now, but I don’t have an ETA for you.

 
HTH,
Ann

2 Likes

The feature multiple code variants analysis was introduced with SonarQube 10.1 starting in Developer Edition. There is an excellent introduction to it here.
You can now have a unified view of a project built for multiple platforms. This first step works only for cases where you can cross-build from one host to all the target platforms.
This first iteration does not handle multiple aspects:

  • There is no way to specialize quality gates per target platform (code variant) for this sort of project
  • You cannot use this with parallel builds/analysis. This is the second iteration that we are collecting interest for here.
1 Like