Possibility to try out a quality profile

SonarQube Cloud Product Manager for a Day
1

Hello!
We would like to try out a quality profile with new rules.

The idea is as follows:

  • we would like to enable some extra rules and evaluate their usefulness and correctness of our settings for them before promoting them to the main branch.
  • we would like to avoid messing up with statistics of the main branch.

In other systems like Checkstyle it’s pretty easily achievable since the configuration is committed in the code and we can easily tune the rule set in the specific branch and evaluate.

Is it possible to achieve in SonarQube Cloud? E.g. by providing a custom quality profile to Maven sonar scanner for a specific branch?

I found an old topic where a similar question raised (Quality profile specific to branches) yet maybe something has changed since then.

1 Like
3

Hi @lrozenblyum,

Thank you for your message!

I believe nothing has changed regarding the ability to use quality profiles specific to branches in SonarQube Cloud, and I am not aware of any planned developments on this point. However, you have several ways to conduct tests:

  1. Use a local SonarQube test instance: You can install SonarQube locally and create a test environment to try out your new quality profiles and rules. This will allow you to experiment without affecting the statistics of your main branch.
  2. Modify the quality profile to test before the analysis and restore it afterward: You can use API calls for this, which will allow you to temporarily change the quality profile based on your needs.
  3. Perform analyses by changing the project key: By creating a separate project with a different key, you can conduct analyses without influencing your main branch.

Please let me know if you need any further information or assistance with these options!

Abdel

2 Likes
6

Thank you for the ideas!

They all are manageable yet also have some drawbacks

  1. Use a local SonarQube test instance Requires pretty big efforts to set up and port all needed data from Cloud
  2. Modify the quality profile to test before the analysis and restore it afterward. It’s what we actually tried to do but due to the fact that our main branch is very active we anyway spoilt its statistics during experiments (because even short term quality profile change was taken into account during simultaneous pushes to the main branch triggering Sonar analyzing).
  3. Perform analyses by changing the project key. Looks a little bit problematic for LoC point of view (our main project is using the bit part of LoC limit in current payment plan) so copying it would require a significant extra payment.

Most likely in the future we may try 2 with temporily blocking merges to the main branch (not ideal but…).

If SonarQube Cloud allow using a quality profile per branch that would be great!

How we see it (inspired by other tools like Checkstyle): rules are also code and thus they can be versioned/branch-specific (thus possible to tried out easily).

Actually SonarQube Cloud already allows suppression of rules on per-branch basis (we can do it in main pom.xml for example); however defining full set of them - would provide another level of flexibility.

1 Like
7

This is a great discussion and having more flexible gates, including different gates on each branch is something we are actively looking into.

We’ll fold your feedback into the discussion, so thank you both!

We don’t have a firm date for this just yet.

3 Likes
12

Thank you for the feedback @lrozenblyum and contributions from @Bachri_Abdel and @john.clifton in the discussion.

I see few opportunities to add to the list John mentioned above:

  1. Allow some rules in a quality profile to be only for reporting and not affect metrics computing or quality gate status.
  2. Have different quality profiles on different branches
  3. Provide rule upgrade impact visibility and cushion while analyzers are updated.

We are actively looking into the third one. We understand that sudden introduction of new rules into quality profile can cause friction. We are aiming to address this problem.

3 Likes
13

Hello @vivek.reghunath,

I find it very interesting to consider the impact of rule updates on analyses.
With each version upgrade, I regularly face the implications of new rules on thousands of DevOps pipelines. We have developed a strategy using staging environments to manage this.
However, it would be so much better to have a dedicated feature in SonarQube. This could help us upgrade the Sonar server version more quickly and with less effort.

I’m glad to see you’re already looking into this issue!

1 Like