we are using Sonar 9.9.5, sonar maven plugin version 4.0.0.4121.
Since we updated our scans to use the scanAll parameter we noticed that we suddenly have Sonar issues in test code located in src/test/…
For java (and other languages?) only special rules created and labeled for test code are applied to this code. For SQL files we now have issues from rules which are not tagged as “test” for files located in src/test/…
I consider this as a bug. Is there some configuration I can do to make it work as expected or is there some other explanation for this?
what do you mean by “you passed these CML and PL/SQL files in to sonar.tests”?
We are using scanAll in maven projects, which I understood means that every file not in src/test/… is scanned as sources. Everything in src/test/… is treated as test files automatically.
Due to the documentation different analysis rules are applied to sources and tests. I thought I could identify the rules applied to test code by checking for the “tests” tag in Sonarqube.
From your answer I understand that this is not the case. Where can I look up which rules are applied to which kind of code?
Does this mean you have XML and PL/SQL files in src/test? And if so, why?
Unfortunately, there’s no good way to search on this.
Let me back up a moment. I asked for examples of rules not tagged with ‘test’ where you saw issues raised on test files. You gave me these two examples:
To be clear, you’re saying you have a PL/SQL file under src/test where a “normal” rule raised an issue?
Does this mean you have XML and PL/SQL files in src/test ? And if so, why?
Yes in some projects we have XML, SQL an other kinds of files in src/test. Developers put them there probably because they belong to some kind of test.
We use scanAll because our maven projects are not pure java projects and also contain sources of other languages.
To be clear, you’re saying you have a PL/SQL file under src/test where a “normal” rule raised an issue?
What is a “normal” rule? Do you mean a rule from a different language like Java? Are xml:S125 and plsql:DeleteOrUpdateWithoutWhereCheck not “normal” rules?
I noticed that some rules (which I could not identify to be tagged as meant for test code) raised issues. So far I did not see any Java issues in these directories. Seems to be the case only for languages like XML, SQL, don’t know what else…
By ‘normal’ rule, I meant one that’s supposed to run on source files, rather than on test files.
xml:S125 is has an analysis scope of ‘all sources’
So I would expect it to run on test files (altho, again, I’m not sure there’s really a concept of test files in the XML analyzer).
On the other hand, the PL/SQL rule says it’s targeted at ‘main sources’
So theoretically, it shouldn’t run on test files. Altho for some languages - and this is likely one - that distinction is largely aspirational, at best. Given that we probably never anticipated tests being written in PL/SQL, and thus never anticipated finding PL/SQL files tucked under src/tests, I’m not shocked that we would be running this rule on “test” files.
Yes. This makes sense. If you’re disturbed by these issues, then you may want to set something like sonar.tests.exclusions=**/*.xml, **/*.plsql. Note that that won’t prevent Java rules that are intended to run on tests from raising issues.
we are currently using sonar.exclusions=src/test/**/* in the project where we faced the issues.
I tried to update our exclusion to your proposal but if I use sonar.tests.exclusions=**/*.sql the issues are still reported. It only works with sonar.exclusions=src/test/**/*.sql
So, first, I fell into one of the classic blunders here:
The property to specify test files is sonar.tests (plural) but the correct property to specify test exclusions is sonar.test.exclusions (singular). (And I knew that. No idea where my head was. )
So… it’s not surprising that that had no effect.
And beyond that… My assumption had been that with scanAll enabled, everything under src/test would be considered a test file. But I checked with a colleague and he pointed out that by default, Maven only recognizes src/test/java as the test directory. So it becomes a question of how scanAll is implemented.
Would you mind running an analysis with -Dsonar.scanner.dumpToFile=out.properties and posting the redacted-as-necessary file so we can see?
I also know that it has to be sonar.test.exclusions and not sonar.tests.exclusions but for some reason I did not think just copy/paste
After doing it the correct way, it still does not exclude as expected. Issues in .sql files are shown.
Can you give me a location where I can upload the out.properties for you? Or maybe send by email? I don’t like the idea of pasting it here publicly readable.
Your properties file confirms that files in src/test/resources are considered “source” files.
For clarity, would your expectation be that everything under src/test be considered a test file? Alternately, is there anything under src/test that “normal” rules should be applied to?
sorry for late answer.
Yes I would expect that everything under src/test is considered a test file and no “normal” rules are applied to any file in src/test.
As a workaround I have already configured our scans to work as expected. We have in the meanwhile excluded the src/test/xyz directories with unexpected results from the scan.
Thank you again for your help and clarification regarding this topic.
)
