Can Build wrapper handle parallel Jobs?

SonarQube Server / Community Build
1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • what are you trying to achieve
  • what have you tried so far to achieve this
    we are using 8.9.6 enterprise edition. And one of our user wants to perform parallel processing with their build wrapper. they are using SQ build wrapper (it’s a GNU C build).
    currently all their projects are building it together and taking longer time to run.
2

Hey there.

Maybe your user can sign-up on this Community directly and ask their question – with more details, like what commands they have tried to run that are/aren’t working.

3

Hi folks,

a bit more in-depth knowledge is needed indeed.
The firmware we are building is splitted up in parallel builds to speed up the build.
One example:
We have ~10 platforms with 2 configurations each. They all run in parallel in different ADO agent jobs.
In this manner we can save app. 95% of the time that is needed for building all targets on one agent sequentially.

We use the SQ BuildWrapper during our build and build it all sequentially at the moment which delays us a lot tbh.

I tried to split our SonarQube job and had the idea to append the generated .json files.
Is this possible somehow?

This is my “vision”:

  • Having all builds running in parallel using the build wrapper
  • Push all generated build files to one agent afterwards
  • Run the scan on the created JSON files and source code (all in the same repo)

I hope you can help me with that :slight_smile:

Best Regards,
Kevin

5

Hi @kevin.tmp and welcome!
Just to be sure I get it right, are you building different code on each parallel run or are you building the same code for 10 x 2 different platform/configuration combination?

7

Hi Fred,

the codebase is the same but we have different projects for each platform.
In those C projects we are using a .c/.h file or not, depending on our demands for that platform.

So the tree is something like:

  • projects

    • platform 1
    • platform 2
      …
    • platform 10

  • source

    • application
      • somethingFancy.c
      • somethingFancy.h
        …
    • bootloader
      • bl.c
      • bl.h

…something like that :smiley:

8

Hi @kevin.tmp ,

At the moment, we don’t support multiple platform/configuration analysis on the same code. In your current monothreaded process, each run is probably overwriting the result of the previous one and you only get the results of the last one.
So you should get an identical result a lot faster by running all build in parallel and wrapping only one of them with the build-wrapper.
Or you can create one SonarQube project per configuration per platform in which case they can all run in parallel, but you will have a lot of common issues between all the analysis.

9

Okay, thank you very much for your help.

I hope this will be implemented one day.

10

You can vote for this feature, and provide additional information on your architecture here: https://portal.productboard.com/sonarsource/3-sonarqube/c/315-analyze-multiple-code-variants