We would like to be able to test quality profiles without changing the currently-selected project profile.
When we wish to add new rules to a quality profile, it is important for us to test those rules on various projects before enabling them. SonarQube is used as a security and code quality auditing tool and running test rules on the primary profile can block merges, skew reports, and generally cause confusion. We need a fluid way to test quality profile modifications.
We are currently able to create a test project (suffix project key with “-TEST”), assign the appropriate test profile(s) there, and then run manual scans against this project. This avoids the critical issues listed above, but it is difficult, time-consuming, and leads to duplicate LOC usage against our license unless we manually modify branch settings.
My first thought was to re-add an analysis parameter similar to “sonar.profile” to allow the analyzer to select the test profile, but [SONAR-5370] Deprecate usage of "sonar.profile" as an analysis parameter - SonarSource lays out some good reasons for its removal several years back.
Instead, it would be helpful to assign a set of “Sampling Quality Profiles” (name?) whose rules would be run in addition to the standard Quality Profiles. Execution of these rules could be hidden behind a
sonar.samplingAnalysis=true property. All violations raised would be given a status and resolution of “Sampled” and would be ignored for metrics and hidden by default in issues lists. Not sure what limitations currently exist in the scanner logic, but it would be helpful if duplicate issues could be raised for rules which exist in both the standard and sampling quality profiles, since those rules may have different configurations.