Sonarqube coverage show 0% (python)

Hello
I’ve been going crazy for a few days,
I do unit tests with pytest (I am a python developer).
I did several steps in the circleci in one job.
the pytest generates the files I see them when I do a cat in the script of the generated files (python step).
on the other hand in the sonarqube interface I always have zero coverage.
help me please

sonar.properties file :

sonar.host.url=https://sonarqube.ifsalpha.com

sonar.projectKey=@APPLICATION_NAME@
sonar.projectName=@PROJECT_SHORTNAME@:@APPLICATION_NAME@
sonar.projectVersion=1.0
sonar.sources=src/filetransformer
sonar.sourceEncoding=UTF-8
sonar.tests=src/tests
sonar.verbose=true

sonar.language=py
sonar.python.pylint=/usr/local/bin/pylint
sonar.python.coverage.reportPath=src/tests/test-results/coverage.xml
sonar.python.xunit.reportPath=src/tests/test-results/results.xml
sonar.coverage.dtdVerification=false

sonar.exclusions=src/**/*.csv, src/**/**/*.csv, src/filetransformer/bypass/**

scan_sonar.sh file:

#!/bin/bash
set -e
set -x
set -o pipefail

export SONAR_URL=https://sonarqube.ifsalpha.com


# PREPARE SONAR CONFIGURATION FILE
echo "BEGIN PREPARE SONAR SCANNER CONFIG FILE"
sed -i s#@PROJECT_SHORTNAME@#${PROJECT_SHORTNAME}#g sonar.properties
sed -i s#@APPLICATION_NAME@#${APPLICATION_NAME}#g sonar.properties
sed -i s#@SONAR_URL@#${SONAR_URL}#g sonar.properties
cat sonar.properties
echo "BEGIN PREPARE SONAR SCANNER CONFIG FILE"



# EXECUTE SONAR SCANNER
echo "BEGIN EXECUTE SCAN SONAR"
sonar-scanner -X -Dproject.settings=sonar.properties -Dsonar.login=${SONAR_LOGIN}
echo "END EXECUTE SCAN SONAR"

# CHECK SONAR SCANNER STATUS
echo "BEGIN CHECK SONAR SCAN STATUS"
echo $SONAR_LOGIN
echo $SONAR_URL
echo $PROJECT_SHORTNAME
echo $APPLICATION_NAME
qualityStatus=$(curl -u ${SONAR_LOGIN}: -H "Accept: application/json" -H "Content-Type: application/json" -X GET "${SONAR_URL}/api/measures/component?component=${APPLICATION_NAME}&metricKeys=alert_status" | jq -r .component.measures[0].value)
echo $qualityStatus
if [[ ${qualityStatus} == "OK" ]]; then
  echo "Quality is OK"
elif [[ ${qualityStatus} == "WARN" ]]; then
  echo "Quality is WARN"
else
  echo "Quality is KO"
  exit 12
fi
pwd
ls -al
sudo apt-get install tree
sudo tree src

scan_python.sh file

#!/bin/bash
SRC_PATH='src/tests'
sudo apt install pip
sudo pip install -r scripts/requirements.txt
sudo pip install -U pytest
sudo pip install -U pytest-cov
sudo pip install -U coverage
cd $SRC_PATH
# PREPARE EXECUTION
rm -rf test-results
mkdir -p test-results

#pytest --cov=src --cov-report html --junitxml=./coverage.xml
pytest -v -o junit_family=xunit1 --cov=../../src/filetransformer --cov-report xml:test-results/results.xml --cov-report html:test-results/cov_html  --junitxml=test-results/coverage.xml
echo '$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$'
sleep 2
chmod 777 test-results/coverage.xml
chmod 777 test-results/results.xml
pwd

ls -al test-results/coverage.xml
ls -al test-results/results.xml
#readlink -f src/tests/coverage.xml
echo '$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$#'
cat test-results/results.xml
ls -al
sleep 3
#sed -i -e  's/workspace\/src\/filetransformer/workspace/g'  test-results/results.xml
sed -i -e  's/\/home.*filetransformer/workspace\/src\/filetransformer/g'  test-results/results.xml

echo '############################'
cat test-results/results.xml

result of executed code python_scan.sh :
( I delete the installation parts)
============================= test session starts ==============================
platform linux – Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 – /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /home/********/workspace/src/tests
plugins: cov-2.10.0
collecting …
collecting 6 items
collected 20 items

    test_parse_params_for_replace.py::test_should_parse_nominal_input PASSED [  5%]
    test_parse_params_for_replace.py::test_should_parse_input_without_output PASSED [ 10%]
    test_parse_params_for_replace.py::test_should_parse_input_without_curly_braces PASSED [ 15%]
    test_parse_params_for_replace.py::test_should_parse_input_without_input PASSED [ 20%]
    test_parse_params_for_replace.py::test_should_not_parse_input_without_commas PASSED [ 25%]
    test_pivot_process_with_date_rule.py::test_should_replace_the_first_word_in_the_list PASSED [ 30%]
    test_pivot_process_with_equal_rule.py::test_should_replace_the_first_word_in_the_list PASSED [ 35%]
    test_pivot_process_with_equal_rule.py::test_should_replace_the_second_word_in_the_list PASSED [ 40%]
    test_pivot_process_with_equal_rule.py::test_should_be_sensitive_to_the_case PASSED [ 45%]
    test_pivot_process_with_equal_rule.py::test_should_not_replace_a_word_not_in_the_list PASSED [ 50%]
    test_pivot_process_with_equal_rule.py::test_should_not_replace_a_word_in_another_column PASSED [ 55%]
    test_pivot_process_with_in_rule.py::test_should_replace_the_first_word_in_the_list PASSED [ 60%]
    test_pivot_process_with_in_rule.py::test_should_replace_the_second_word_in_the_list PASSED [ 65%]
    test_pivot_process_with_in_rule.py::test_should_not_be_sensitive_to_the_case PASSED [ 70%]
    test_pivot_process_with_in_rule.py::test_should_not_replace_a_word_not_in_the_list PASSED [ 75%]
    test_pivot_process_with_in_rule.py::test_should_not_replace_a_word_in_another_column PASSED [ 80%]
    test_pivot_process_with_substr_rule.py::test_should_work_for_the_first_substring_in_the_list PASSED [ 85%]
    test_pivot_process_with_substr_rule.py::test_should_not_work_if_not_in_the_list PASSED [ 90%]
    test_pivot_process_with_substr_rule.py::test_should_replace_the_fourth_word_in_the_list PASSED [ 95%]
    test_pivot_process_with_substr_rule.py::test_should_not_be_sensitive_to_the_case PASSED [100%]

    =============================== warnings summary ===============================
    ../filetransformer/tools_bypass.py:275
      ../filetransformer/tools_bypass.py:275: DeprecationWarning: invalid escape sequence \[
        """

    ../filetransformer/tools_bypass.py:605
      ../filetransformer/tools_bypass.py:605: DeprecationWarning: invalid escape sequence \(
        m = re.match('REPLACE\({?(?P<column_name>\w+)}?, "*(?P<value_to_replace>\w*)"*, "*(?P<value_output>\w*)"*\)', output_value)

    test_pivot_process_with_equal_rule.py::test_should_replace_the_first_word_in_the_list
    test_pivot_process_with_equal_rule.py::test_should_replace_the_second_word_in_the_list
    test_pivot_process_with_equal_rule.py::test_should_be_sensitive_to_the_case
    test_pivot_process_with_equal_rule.py::test_should_not_replace_a_word_not_in_the_list
    test_pivot_process_with_in_rule.py::test_should_replace_the_first_word_in_the_list
    test_pivot_process_with_in_rule.py::test_should_replace_the_second_word_in_the_list
    test_pivot_process_with_in_rule.py::test_should_not_be_sensitive_to_the_case
    test_pivot_process_with_in_rule.py::test_should_not_replace_a_word_not_in_the_list
      /usr/local/lib/python3.6/site-packages/pandas/core/indexing.py:205: SettingWithCopyWarning: 
      A value is trying to be set on a copy of a slice from a DataFrame
      
      See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
        self._setitem_with_indexer(indexer, value)

    -- Docs: https://docs.pytest.org/en/stable/warnings.html
    - generated xml file: /home/********/workspace/src/tests/test-results/coverage.xml -

    ----------- coverage: platform linux, python 3.6.8-final-0 -----------
    Coverage HTML written to dir test-results/cov_html
    Coverage XML written to file test-results/results.xml

    ======================= 20 passed, 10 warnings in 1.07s ========================
    $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
    /home/********/workspace/src/tests
    -rwxrwxrwx 1 ******** ******** 3894 Aug  5 13:16 test-results/coverage.xml
    -rwxrwxrwx 1 ******** ******** 35873 Aug  5 13:16 test-results/results.xml
    $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$#
    total 96
    drwxr-xr-x  6 ******** ********  4096 Aug  5 13:16 .
    drwxr-xr-x 14 ******** ********  4096 Aug  5 13:15 ..
    -rw-r--r--  1 ******** ******** 53248 Aug  5 13:16 .coverage
    drwxr-xr-x  3 ******** ********  4096 Aug  5 13:16 .pytest_cache
    -rw-r--r--  1 ******** ********     0 Aug  5 13:15 __init__.py
    drwxr-xr-x  2 ******** ********  4096 Aug  5 13:16 __pycache__
    drwxr-xr-x  3 ******** ********  4096 Aug  5 13:16 test-results
    -rw-r--r--  1 ******** ********  1437 Aug  5 13:15 test_parse_params_for_replace.py
    -rw-r--r--  1 ******** ********  1417 Aug  5 13:15 test_pivot_process_with_date_rule.py
    -rw-r--r--  1 ******** ********  2690 Aug  5 13:15 test_pivot_process_with_equal_rule.py
    -rw-r--r--  1 ******** ********  2684 Aug  5 13:15 test_pivot_process_with_in_rule.py
    -rw-r--r--  1 ******** ********  3450 Aug  5 13:15 test_pivot_process_with_substr_rule.py
    drwxr-xr-x  5 ******** ********  4096 Aug  5 13:15 tmp
    ############################
    Feature: in operator

      Background:  

      Scenario: the first word in the list should be replaced 
        Given a rule
        Given a rule
        And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
        And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
        And this rule outputs on the column "first_input_column" with the value "'mot'"
        And this rule outputs on the column "first_input_column" with the value "'mot'"
        Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
        Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
        When A file with "Mots" in the "first_input_column" is transformed
        When A file with "Mots" in the "first_input_column" is transformed
        Then I get an output file with "mot" in the "first_pivot_column"
        Then I get an output file with "mot" in the "first_pivot_column"

      Scenario: the second word in the list should be replaced 
        Given a rule
        Given a rule
        And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
        And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
        And this rule outputs on the column "first_input_column" with the value "'mot'"
        And this rule outputs on the column "first_input_column" with the value "'mot'"
        Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
        Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
        When A file with "OtherMot" in the "first_input_column" is transformed
        When A file with "OtherMot" in the "first_input_column" is transformed
        Then I get an output file with "mot" in the "first_pivot_column"
        Then I get an output file with "mot" in the "first_pivot_column"

      Scenario: a complete test 
        Given a rule
        Given a rule
        And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
        And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
        And this rule outputs on the column "first_input_column" with the value "'mot'"
        And this rule outputs on the column "first_input_column" with the value "'mot'"
        Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
        Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
        Given the input columns "first_input_column/second_input_column" associated to the pivot columns "first_pivot_column/second_pivot_column"
        Given the input columns "first_input_column/second_input_column" associated to the pivot columns "first_pivot_column/second_pivot_column"
        When the following file is transformed
          | first_input_column | second_input_column |
          | Mots               | *                   |
          | OtherMot           | *                   |
          | othermot           | *                   |
          | word               | *                   |
          | *                  | OtherMot            |
        When the following file is transformed
          | first_input_column | second_input_column |
          | Mots               | *                   |
          | OtherMot           | *                   |
          | othermot           | *                   |
          | word               | *                   |
          | *                  | OtherMot            |
        Then I get the following file
          | first_pivot_column | second_pivot_column |
          | mot                | *                   |
          | mot                | *                   |
          | mot                | *                   |
          | word               | *                   |
          | *                  | OtherMot            |
        Then I get the following file
          | first_pivot_column | second_pivot_column |
          | mot                | *                   |
          | mot                | *                   |
          | mot                | *                   |
          | word               | *                   |
          | *                  | OtherMot            |

    1 feature passed, 0 failed, 0 skipped
    3 scenarios passed, 0 failed, 0 skipped
    19 steps passed, 0 failed, 0 skipped, 0 undefined
    Took 0m0.033s

this is a begin part of result.xml file :

<?xml version="1.0" ?>
<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="0.1847" lines-covered="176" lines-valid="953" timestamp="1596620696116" version="5.2.1">
	<!-- Generated by coverage.py: https://coverage.readthedocs.io -->
	<!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
	<sources>
		<source>/home/********/workspace/src/filetransformer</source>
	</sources>
	<packages>
		<package branch-rate="0" complexity="0" line-rate="0.1847" name=".">
			<classes>
				<class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
					<methods/>
					<lines/>
				</class>
				<class branch-rate="0" complexity="0" filename="app.py" line-rate="0" name="app.py">
					<methods/>
					<lines>
						<line hits="0" number="1"/>
						<line hits="0" number="2"/>
						<line hits="0" number="3"/>
						<line hits="0" number="4"/>
						<line hits="0" number="5"/>
						<line hits="0" number="6"/>
						<line hits="0" number="7"/>
						<line hits="0" number="8"/>
						<line hits="0" number="9"/>
						<line hits="0" number="10"/>
						<line hits="0" number="11"/>
						<line hits="0" number="12"/>
						<line hits="0" number="13"/>
						<line hits="0" number="14"/>
						<line hits="0" number="15"/>
						<line hits="0" number="16"/>
						<line hits="0" number="19"/>
						<line hits="0" number="21"/>
						<line hits="0" number="22"/> 

I try the sed command ( maybe problem with absolute path ? )

sed -i -e 's/\/home.*filetransformer/workspace\/src\/filetransformer/g' test-results/results.xml

an this is the begin of file :

<?xml version="1.0" ?>
<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="0.1847" lines-covered="176" lines-valid="953" timestamp="1596620696116" version="5.2.1">
	<!-- Generated by coverage.py: https://coverage.readthedocs.io -->
	<!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
	<sources>
		<source>workspace/src/filetransformer</source>
	</sources>
	<packages>
		<package branch-rate="0" complexity="0" line-rate="0.1847" name=".">
			<classes>
				<class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
					<methods/>
					<lines/>
				</class>
				<class branch-rate="0" complexity="0" filename="app.py" line-rate="0" name="app.py">
					<methods/>
					<lines>
						<line hits="0" number="1"/>
						<line hits="0" number="2"/>
						<line hits="0" number="3"/>
						<line hits="0" number="4"/>
						<line hits="0" number="5"/>
						<line hits="0" number="6"/>
						<line hits="0" number="7"/>
						<line hits="0" number="8"/>
						<line hits="0" number="9"/>
						<line hits="0" number="10"/>
						<line hits="0" number="11"/>
						<line hits="0" number="12"/>
						<line hits="0" number="13"/>
						<line hits="0" number="14"/>
						<line hits="0" number="15"/>
						<line hits="0" number="16"/>
						<line hits="0" number="19"/>
						<line hits="0" number="21"/>
						<line hits="0" number="22"/>
						<line hits="0" number="23"/>
						<line hits="0" number="24"/>
						<line hits="0" number="25"/>
						<line hits="0" number="27"/>
						<line hits="0" number="28"/>

Hi @nefissi and welcome to the community,

Could you give us your version of SonarQube? The comments bellow apply for the last version of our python analyzer.

The first thing I noticed is that your configuration file might not be named correctly. It should be sonar-project.properties, not sonar.properties.

The next problem is that the property sonar.python.coverage.reportPaths in your sonar.properties file is missing an s at the end. The property sonar.python.coverage.reportPath previously existed but it has been removed.

It also seems that the coverage and test reports are inverted. The option --junitxml=test-results/coverage.xml generates a report listing the tests which are ran, if they succeeded, their execution time, etc… and it is misnamed coverage.xml. The option --cov-report xml:test-results/results.xml generates a coverage report named results.xml.
Thus you should instead use the options --cov-report xml:test-results/coverage.xml and --junitxml=test-results/results.xml

Let me know if this fixes your problem.

1 Like

Thnak you Nicolas your comment resolv my problem

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.