Sonar-scanner not scanning test folder

Versions

  • SonarQube server: * Developer Edition * Version 9.5 (build 56709)
  • Scanner Version:
    $ sonar-scanner --version
    INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
    INFO: Project root configuration file: NONE
    INFO: SonarScanner 4.7.0.2747
    INFO: Java 11.0.15 Debian (64-bit)
    INFO: Linux 4.18.0-383.el8.x86_64 amd64
    

Trying to achieve:
Trying to scan source and test files as part of the process to add code coverage.
If I navigate to the project report I see that the test files are not scanned by SonarQube.
My project follows the exact structure as in the simple example on the page Narrowing the Focus

(root)
   - src
   - test

The following is set in our sonar-project.properties file

sonar.sources=src/
sonar.tests=test/

When I run the analysis cli I do not see that the files in the test folder gets analyzed.
When I build using the build-wrapper, I do see these test files getting compiled and the test executables are created.

What I have tried
If I change the properties as follow:

sonar.sources=./

I see the test files getting scanned.

If I change it to:

sonar.sources=./
sonar.tests=test/

I get the following error when trying to run the cli:

Error during SonarScanner execution
ERROR: File test/CMakeLists.txt can't be indexed twice. Please check that inclusion/exclusion patterns produce disjoint sets for main and test files

Any help will be appreciated.

I could get my coverage reported using gcovr but SonarQube still reports zero unit tests.

SQ_Coverage

Hi,

Could you clarify what you’re trying to accomplish? Is it that you want issues raised on your test code? Because the fact that you’re not seeing many (any?) issues raised on your test code does not mean that it’s not being included in the analysis. We just don’t have that many rules that run on test code.

Regarding your unit test count, are you passing it in as part of your test execution data?

 
Ann

Hi,

Yes I would like the test code to be scanned.

It might be, if it does I do however expect that the CLI indicates that it scans the test files. Is this expectation incorrect?

My CLI log is:

...
INFO: Sensor CFamily [cpp]
INFO: CFamily plugin version: 6.34.0.48468
INFO: Using build-wrapper output: /home/Buildit/my_project/build/bw-output/build-wrapper-dump.json
INFO: Available processors: 72
INFO: Using 4 threads for analysis, according to value of "sonar.cfamily.threads" property.
INFO: Loading cache from: server
INFO: [pool-3-thread-1] /home/Buildit/my_project/src/file_01.cpp
INFO: [pool-3-thread-2] /home/Buildit/my_project/src/file_02.cpp
INFO: [pool-3-thread-3] /home/Buildit/my_project/src/file_03.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/src/file_04.cpp
INFO: [pool-3-thread-3] /home/Buildit/my_project/src/file_05.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/src/file_06.cpp
INFO: [pool-3-thread-3] /home/Buildit/my_project/src/file_07.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/src/file_08.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/src/file_09.cpp
INFO: [pool-3-thread-2] /home/Buildit/my_project/src/file_10.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/src/file_11.cpp
INFO: [pool-3-thread-2] /home/Buildit/my_project/src/main.cpp
INFO: PCH: unique=0 use=0 (forceInclude=0,throughHeader=0,firstInclude=0) out of 12 (forceInclude=0,throughHeader=0)
INFO: SE: 12 out of 12
INFO: Z3 refutation rate: 0 out of 0
INFO: Subprocess(es) done in 102479ms
INFO: Cache: 0/12 hits, 253591 bytes
INFO: 12 compilation units analyzed
INFO: Sensor CFamily [cpp] (done) | time=104789ms
...

On the server dashboard on the “Code” page I get the following:

When I change to sonar.sources=./ and just rerun sonar-scanner the CLI is

...
INFO: Sensor CFamily [cpp]
INFO: CFamily plugin version: 6.34.0.48468
INFO: Using build-wrapper output: /home/Buildit/my_project/build/bw-output/build-wrapper-dump.json
INFO: Available processors: 72
INFO: Using 4 threads for analysis, according to value of "sonar.cfamily.threads" property.
INFO: Loading cache from: server
INFO: Cache hit for: /home/Buildit/my_project/src/file_01.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_02.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_03.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_04.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_05.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_06.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_07.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_08.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_09.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_10.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/file_11.cpp
INFO: Cache hit for: /home/Buildit/my_project/src/main.cpp
INFO: [pool-3-thread-1] /home/Buildit/my_project/test/file_01.cpp
INFO: [pool-3-thread-2] /home/Buildit/my_project/test/file_02.cpp
INFO: [pool-3-thread-3] /home/Buildit/my_project/test/file_03.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/test/file_04.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/test/file_05.cpp
INFO: [pool-3-thread-1] /home/Buildit/my_project/test/file_06.cpp
INFO: [pool-3-thread-2] /home/Buildit/my_project/test/file_07.cpp
INFO: [pool-3-thread-3] /home/Buildit/my_project/test/file_08.cpp
INFO: [pool-3-thread-1] /home/Buildit/my_project/test/file_09.cpp
INFO: [pool-3-thread-2] /home/Buildit/my_project/test/file_10.cpp
INFO: [pool-3-thread-4] /home/Buildit/my_project/test/main.cpp
INFO: PCH: unique=0 use=0 (forceInclude=0,throughHeader=0,firstInclude=0) out of 23 (forceInclude=0,throughHeader=0)
INFO: SE: 23 out of 23
INFO: Z3 refutation rate: 8 out of 9
INFO: Subprocess(es) done in 43465ms
INFO: Cache: 12/23 hits, 581769 bytes
INFO: 23 compilation units analyzed
...

On the server dashboard on the “Code” page I get the following:

Thus my conclusion is that with the original setup the tests are not getting scanned:

sonar.sources = src/
sonar.tests   = test/

Unit test count:

Regarding your unit test count, are you passing it in as part of your test execution data?

Thanks for the information. Since it is unrelated to the issue (test code not getting scanned) we can ignore this aspect in this thread. When I get to work on this and there are issues I will log a new issue.

Hi,

What you’re seeing is perfectly normal behavior.

I guess you’re basing this on the fact that no issues or Lines of Code are detected for your test files with the original setup? The fact is that we just don’t have many (any?) rules for tests in CFamily languages. It’s perfectly normal that we woudln’t detect “lines of code” in test files because… they’re lines of tests, not production code. Here’s a screenshot from a part of SonarQube itself:

You’ll see that there are no Lines of Code for these tests either. It’s in Java, so there are a few specialized rules that run on tests, hence the issues.

If you want normal rules run on your tests, then you’ll need to include them in the sonar.sources configuration. Note that this will affect your license LOC usage.

Does this help? Answer the question?

 
Ann

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.