How can i remove test files from files to cover?

[doi_zece]

Hi,
I’m trying to integrate my python project with sonarqube, I manage to get it working but, i have a problem with the test file. Basically they are show as not covered properly in the sonar console.
My question is: How can i exclude the test files from the files to cover bucket?

I tried a couple of configuration, but so far no luck. If i put the test file in the excluded list, i get 0% coverage on all the project files. I get the same results if i exclude the test files from the sources or from the test coverage.

Any idea how i can exclude the tests files from coverage?

My CI stages are

test-binaries:
  <<: *docker
  stage: test
  when: manual
  script:
    - apk add --no-cache py-pip
    - cp version.txt server/
    - pip install docker-compose==$DOCKER_COMPOSE
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker load -i image_artifacts/middleware_image.tar
    - docker tag $WEB_SERVER_IMAGE:$VERSION $WEB_SERVER_IMAGE_NAME
    - cd server/tests
    - docker-compose build
    - docker-compose run --rm initdb
    - docker-compose run --rm pytest --reruns 5 --reruns-delay 1 --cov bolt --cov tests --cov-report xml:cov.xml --junitxml=junit-report.xml
    - cd ../..
  artifacts:
    paths:
      - server/cov.xml
      - server/junit-report.xml
    expire_in: 1 week
  allow_failure: true

and sonar

sonar scan:
  <<: *docker
  stage: codequality
  script:
    - apk add --no-cache openssl ca-certificates curl unzip openjdk8-jre
    - apk add --update-cache --upgrade curl
    - curl -L https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-${SONAR_SCANNER_CLI}-linux.zip -o /opt/sonar.zip
    - unzip /opt/sonar.zip -d /opt
    - mv /opt/sonar-scanner-${SONAR_SCANNER_CLI}-linux/jre/bin/java /opt/sonar-scanner-${SONAR_SCANNER_CLI}-linux/jre/bin/java_org
    - ln -s /usr/lib/jvm/default-jvm/jre/bin/java /opt/sonar-scanner-${SONAR_SCANNER_CLI}-linux/jre/bin/java
    - export PATH="$PATH:/opt/sonar-scanner-${SONAR_SCANNER_CLI}-linux/bin"
    - sed 's/filename\=\"/filename\=\"\/builds\/core-tech\/tools\/nlu\/mix-nlu-middleware\/server\/tests\//g' ./server/cov.xml >> ./server/covu.xml
    - cp ./server/covu.xml ./server/tests/covu.xml
    - cp ./server/junit-report.xml ./server/tests/junit-report.xml
    - sonar-scanner -Dsonar.projectKey=mix-nlu-middleware -Dsonar.sources=./server -Dsonar.host.url=$SONAR_SERVER_HOST -Dsonar.login=$SONAR_LOGIN -Dsonar.python.coverage.reportPaths=server/tests/covu.xml -Dsonar.junit.reportPaths=server/tests/junit-report.xml -Dsonar.exclusions=server/bolt/core/db/migrations/versions/**,server/**/__init__.py,server/bolt/api/onse/ontology/**,templates/**,server/scripts/**,server/setup.py,server/bolt/api/routing.py
  when: manual

covu.txt (1.4 MB)

[ganncamp (G Ann Campbell)]

Hi,

You’ve set this on your analysis command line:

-Dsonar.sources=./server

So everything under that directory is treated as a source file & thus in need of coverage. You can remedy this with a more selective definition of your source file locations

 
HTH,
Ann

[doi_zece]

@ganncamp

./server

I’ve tried already -Dsonar.sources=./server/bolt but in that situation I get 0% coverage

[ganncamp (G Ann Campbell)]

Hi,

I guess you’ve got your test files jumbled into the same directory as your source files? If so, that’s the root of the problem. You’ve said “everything in the server directory is a source file” and SonarQube takes you at your word.

If you don’t want to sort things into subdirectories (I’m not actually a fan of changing the project to make a tool happy) you might be able to work through this with exclusions. I don’t normally suggest combining inclusions and exclusions (gets terribly confusing) but that might be what you need here:

• sonar.sources=server
• sonar.tests=server
• source file exclusion **/*test.py
• test file inclusion **/*test.py

 
HTH,
Ann

[doi_zece]

No, source and test files are in separate folders. You can see that in the snapshot that I’ve attached.
The sources are under /server/bolt while the tests are under /server/tests.
I will try to add sonar source and sonar tests to see if that helps

[ganncamp (G Ann Campbell)]

Hi,

If they’re in separate folders then it’s a lot simpler:

sonar.sources=server/bolt
sonar.tests=server/tests

Note that I’m not starting with / (box root) or ./ (no need).

 
HTH,
Ann

[doi_zece]

It doesn’t work. I’ve added:
-Dsonar.sources=server/bolt -Dsonar.tests=server/tests

sonar-scanner -Dsonar.projectKey=mix-nlu-middleware -Dsonar.sources=server/bolt -Dsonar.tests=server/tests -Dsonar.host.url=$SONAR_SERVER_HOST -Dsonar.login=$SONAR_LOGIN -Dsonar.python.coverage.reportPaths=server/tests/covu.xml -Dsonar.junit.reportPaths=server/tests/junit-report.xml -Dsonar.exclusions=server/bolt/core/db/migrations/versions/,server//init.py,server/bolt/api/onse/ontology/,templates/,server/scripts/**,server/setup.py,server/bolt/api/routing.py

In the sonar console the tests files are still showing as uncovered files, and this time the covu file doesn’t show up on the sonar console either. So i see 0% coverage.

[Andrea_Guarino]

Hi @doi_zece,

In the coverage report you attached it seems like there are also test files listed there.
For example I can see server/tests/api/qws/projects_test.py with its absolute path.
Can you please make sure they don’t show up in the coverage report? You can have a look at our guide about generating coverage report here

If you still encounter problems, can you please attach the analysis logs?

[doi_zece]

@Andrea_Guarino where do I find the analysis logs? Can i download them from the Sonar console or i need access to the sonar system ?

[Andrea_Guarino]

hey @doi_zece,

It’s the output of sonar-scanner command you have configured in your CI. Do you have a way to access it and send it to us?

Also, it would be great if you could activate debug log in sonar-scanner by adding property -Dsonar.log.level=DEBUG

[doi_zece]

@Andrea_Guarino please find the logs attached! analyze.txt (266.4 KB)
covu.txt (86.4 KB)

Note that for quicker turnaround, I’ve reduced the scope of the tests to bolt/admin source folder only. For this particular run, sonar shows 0.0% coverage

[Andrea_Guarino]

hey @doi_zece,

In the logs I can see the following errors:

20:17:10.991 ERROR: Cannot resolve the file path '/builds/core-tech/tools/nlu/mix-nlu-middleware/server/routing.py' of the coverage report, the file does not exist in all <source>.
20:17:10.998 ERROR: Cannot resolve 27 file paths, ignoring coverage measures for those files

Could you please double check that filepaths reported in the coverage file are correct and exist?

[doi_zece]

@Andrea_Guarino that was it. The path was not correct. It works now. Thanks a lot!