Thousands of unwanted issues in test code after upgrading to SonarQube 10

[Thomas_Mauch]

Hi
I’ve seen the announcement

Java Test Code: 40 more rules enabled by default

Hello Java devs, We want to help you write clean test code! However previously, Sonar was only using a limited set of rules on your test code, see the list here. Sonar relies on your configuration to determine which files are the main code and which are tests. Our Maven and Gradle scanners handle this, but it can be configured, see the docs. To improve, we’re reviewing our existing rules for main code to select those that make sense for test code as well. As a first step, we’ve enabled two …

and various questions whether the analysis scope could be controlled per rule, but without answers, e.g.

Rule java:S1874 "@Deprecated" code should not be used - in test code

Also keep in mind that we have hundreds of thousands of users, and having each of them take the time to review 700+ rules seems like a big ask. Sure so the default behaviour would be as it is today - tests are only covered by the existing test-only rules, but you have the option to run most/all rules against tests as well as other code, in a configurable way. You guys reviewing rules and deciding what is/is not noisy from your perspective does not mean that your hundreds of thousands of user…

Is there still no way to control for me as user whether applying a rule make sense for my code base as on test level?

I think it is quite unfortunate if a simple update of Sonar introduces thousands of issues to your codebase.

I would appreciate if it could be controlled on rule level to which scope it should be applied.

Best,
Thomas

[Colin]

Hi @Thomas_Mauch,

Unfortunately, there aren’t any significant updates on this issue. However, I recommend using the Ignore Issues on Multiple Criteria feature at either the global or project level to reduce unnecessary alerts.

For example, to exclude java:S1874 from all test files, you could set up a rule similar to the one shown below:

Hope this helps!

[Thomas_Mauch]

Hi Colin,
Thanks for your answer. Yes, then ignoring the issues like this seems to be the best option. We would still prefer, if we could control per rule in the quality profile whether it should be active for tests or not.
Thanks,
Thomas

[Bachri_Abdel]

HI @Thomas_Mauch ,

Thomas,

I understand your concerns about the impact that SonarQube updates can have on your codebase, especially when it comes to test code. You raise a valid point - it can be quite unfortunate if a simple SonarQube update introduces thousands of new issues, particularly in your test code.

Regarding the management of quality profiles, I avoid using the default or built-in quality profiles to better control the updates to the quality profiles used by my teams. I make it a practice to backup and version the quality profiles in XML format. When upgrading SonarQube, I compare the quality profiles and work with the development teams to ensure the changes proposed by SonarQube are consistent with our internal development standards.

As for the application of rules to production code versus test code, SonarQube currently does not provide the functionality to use different quality profiles for these two code types. However, you have to decide whether your test code should be held to the same quality standards as your production code, or if it should be excluded from the analysis altogether.

If you choose to include the test code, you’ll need to decide whether to keep or exclude certain rules - it will be an all-or-nothing decision for the entire codebase, both production and test.

An other solution then @Colin’s could be to keep certain rules and consider all the issues identified in the test sources as false positives or “won’t fix”.

I hope these suggestions are helpful in managing the SonarQube analysis of your test code. Please let me know if you have any other questions.