I recently accidentally discovered that SonarQube reports “Unused imports” in our non-test Java source files, but does not detect that in our test classes. I’m hearing that SonarQube executes “different rules” on test classes than on non-test classes. I find it hard to believe that it would be considered unimportant for unused imports to be present in test classes, but not ok in non-test classes.
Is this the only “common-sense” rule that is not being applied to test classes?
I’ve been told that this is controlled by “tags” on the rule. If the rule has the “tests” tag, it will apply that rule to test classes.
The thing is, I’ve looked through many of the rules without the “tests” tag, and I see numerous “common-sense” rules about quality Java code that don’t have that tag. It seems extremely ironic that someone decided that typical rules about quality Java code shouldn’t apply to tests.
In talking to our local SonarQube support team, they believe that they have no control over tags on rules.
For the most part, we agree that there are lots of rules that make sense on test code.
And, we want to be thoughtful about how we introduce it:
Are issues on test code as “important” as those on source code? Should they fail the Quality Gate in the same way?
Today Lines of Test Code are not counted towards a license. If most rules start running against test code, should that change?
Of the 5000+ rules, which rules always make sense on test code, sometimes make sense on test code, and never make sense on test code?
These are tough questions and right now, instead of flipping on individual rules when they make sense, want to figure our long term approach. Stay tuned (not tomorrow, but next year).
To lead to an answer to at least the first question, ask yourself this: Is quality of test code as important as quality of the code under test? I’d say it needs to be. In some organizations, test code would be used as documentation, to help developers understand the code under test.
For the rule in question, “unused imports”, if a dependency that someone thought wasn’t used anymore was removed, and unused imports were the only remaining references to that dependency, the build will fail, whether it was in test code or not.
I have no perspective to answer the second question.
For the third question, I’d have to say the answer to the first question helps answer that question.
I found this thread because today I noticed that some stuff in my test code doesn’t get flagged by SonarLint which does get flagged in the production code of my project. At first I assumed there must be something wrong with my SonarLint setup (IntelliJ).
To be honest, differentiating between production code and test code like this doesn’t make a lot of sense to me. Test code is a vital part of a project. It needs to be understandable, maintainable and robust, just like the production code. And if for some reason you need to break the Sonar rules in your test code, you should have to be deliberate about it and use @SuppressWarnings or so. No difference to production code.
So, it doesn’t really seem like such a tough question to me: just don’t make any difference at all between production code and test code when analyzing.
In terms of licensing: you can give users some mechanism to exclude parts of their code base from being analyzed and consequentially not count toward their license. That way they can decide for themselves.
Hi,
I also find the quality of each and every code is important, doesn’t mater if it is a source code or a test code. In fact anyone who wanted to clear their technical debt will find it easier to start with small chunks(say test code). Enabling most of the rules for common for both source and tests will greatly be helpful.
Thanks.
I personally totally agree with what you said: test code should be as clean as the main/prod code. The quality of the test code will impact the quality of the overall software delivered in production.
When we worked to define what is Clean Code, we clarified that “Code” means all kinds of code: test, prod, build scripts, 3rd party code referenced, infra-as-code, … Whatever code that is written by a Dev or DevOps must be clean before the software is released to production.
“Enabling most of the rules for both source and tests” would make sense and this is something we tried in the past. Unfortunately, we did not find an alignment internally at Sonar on the points mentioned by @Colin. We will retry for sure, I don’t know exactly when.