So, after running SonarQube analysis without excluding test files from our analysis, we encountered boolean statements where the developer forgot to prepend an assert keyword. SonarQube reported these lines as bug (Remove or refactor this statement; it has no side effects., python:S905) as the statement in itself is meaningless. In my opinion, this is not only a normal bug but a serious issue as the test case is always green (no matter what happens) as nothing is really asserted and the test is subsequently delusive.
If we’d have excluded the test files right away from analysis, we would not have discovered these flaws on the spot. I don’t really suggest a new rule but rather wanted to raise awareness for this particular issue. I think, it might be a nice feature when SonarQube analyzes test files on lonely boolean statements without asserts by default.
Thanks for your suggestion. Some languages have rules on tests that would have caught this. Just to be specific, I guess you’d like those test rules implemented for Python too?
I didn’t know that SonarQube supports this already for other languages… but yes, in our case, this was only Python related and it would be nice, if these test rules would be analyzed as well for Python code by default
Your suggestion comes at a rather good time. As Ann said, we have rules targeting test code for a few languages and, until today, Python was not one of them.
However, we are currently working on implementing this MMF, whose goal is to help Python developers write better tests. This first batch of rules targeting Python tests should be available in the next version of SonarQube/SonarLint, and in a few weeks on SonarCloud.
It’s still not decided if we will enable the rule S905 on test files or create a dedicated rule for the use case you suggest, but we will certainly discuss this internally.
