Hello
I’ve been going crazy for a few days,
I do unit tests with pytest (I am a python developer).
I did several steps in the circleci in one job.
the pytest generates the files I see them when I do a cat in the script of the generated files (python step).
on the other hand in the sonarqube interface I always have zero coverage.
help me please
sonar.properties file :
sonar.host.url=https://sonarqube.ifsalpha.com
sonar.projectKey=@APPLICATION_NAME@
sonar.projectName=@PROJECT_SHORTNAME@:@APPLICATION_NAME@
sonar.projectVersion=1.0
sonar.sources=src/filetransformer
sonar.sourceEncoding=UTF-8
sonar.tests=src/tests
sonar.verbose=true
sonar.language=py
sonar.python.pylint=/usr/local/bin/pylint
sonar.python.coverage.reportPath=src/tests/test-results/coverage.xml
sonar.python.xunit.reportPath=src/tests/test-results/results.xml
sonar.coverage.dtdVerification=false
sonar.exclusions=src/**/*.csv, src/**/**/*.csv, src/filetransformer/bypass/**
scan_sonar.sh file:
#!/bin/bash
set -e
set -x
set -o pipefail
export SONAR_URL=https://sonarqube.ifsalpha.com
# PREPARE SONAR CONFIGURATION FILE
echo "BEGIN PREPARE SONAR SCANNER CONFIG FILE"
sed -i s#@PROJECT_SHORTNAME@#${PROJECT_SHORTNAME}#g sonar.properties
sed -i s#@APPLICATION_NAME@#${APPLICATION_NAME}#g sonar.properties
sed -i s#@SONAR_URL@#${SONAR_URL}#g sonar.properties
cat sonar.properties
echo "BEGIN PREPARE SONAR SCANNER CONFIG FILE"
# EXECUTE SONAR SCANNER
echo "BEGIN EXECUTE SCAN SONAR"
sonar-scanner -X -Dproject.settings=sonar.properties -Dsonar.login=${SONAR_LOGIN}
echo "END EXECUTE SCAN SONAR"
# CHECK SONAR SCANNER STATUS
echo "BEGIN CHECK SONAR SCAN STATUS"
echo $SONAR_LOGIN
echo $SONAR_URL
echo $PROJECT_SHORTNAME
echo $APPLICATION_NAME
qualityStatus=$(curl -u ${SONAR_LOGIN}: -H "Accept: application/json" -H "Content-Type: application/json" -X GET "${SONAR_URL}/api/measures/component?component=${APPLICATION_NAME}&metricKeys=alert_status" | jq -r .component.measures[0].value)
echo $qualityStatus
if [[ ${qualityStatus} == "OK" ]]; then
echo "Quality is OK"
elif [[ ${qualityStatus} == "WARN" ]]; then
echo "Quality is WARN"
else
echo "Quality is KO"
exit 12
fi
pwd
ls -al
sudo apt-get install tree
sudo tree src
scan_python.sh file
#!/bin/bash
SRC_PATH='src/tests'
sudo apt install pip
sudo pip install -r scripts/requirements.txt
sudo pip install -U pytest
sudo pip install -U pytest-cov
sudo pip install -U coverage
cd $SRC_PATH
# PREPARE EXECUTION
rm -rf test-results
mkdir -p test-results
#pytest --cov=src --cov-report html --junitxml=./coverage.xml
pytest -v -o junit_family=xunit1 --cov=../../src/filetransformer --cov-report xml:test-results/results.xml --cov-report html:test-results/cov_html --junitxml=test-results/coverage.xml
echo '$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$'
sleep 2
chmod 777 test-results/coverage.xml
chmod 777 test-results/results.xml
pwd
ls -al test-results/coverage.xml
ls -al test-results/results.xml
#readlink -f src/tests/coverage.xml
echo '$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$#'
cat test-results/results.xml
ls -al
sleep 3
#sed -i -e 's/workspace\/src\/filetransformer/workspace/g' test-results/results.xml
sed -i -e 's/\/home.*filetransformer/workspace\/src\/filetransformer/g' test-results/results.xml
echo '############################'
cat test-results/results.xml
result of executed code python_scan.sh :
( I delete the installation parts)
============================= test session starts ==============================
platform linux – Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 – /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /home/********/workspace/src/tests
plugins: cov-2.10.0
collecting …
collecting 6 items
collected 20 items
test_parse_params_for_replace.py::test_should_parse_nominal_input PASSED [ 5%]
test_parse_params_for_replace.py::test_should_parse_input_without_output PASSED [ 10%]
test_parse_params_for_replace.py::test_should_parse_input_without_curly_braces PASSED [ 15%]
test_parse_params_for_replace.py::test_should_parse_input_without_input PASSED [ 20%]
test_parse_params_for_replace.py::test_should_not_parse_input_without_commas PASSED [ 25%]
test_pivot_process_with_date_rule.py::test_should_replace_the_first_word_in_the_list PASSED [ 30%]
test_pivot_process_with_equal_rule.py::test_should_replace_the_first_word_in_the_list PASSED [ 35%]
test_pivot_process_with_equal_rule.py::test_should_replace_the_second_word_in_the_list PASSED [ 40%]
test_pivot_process_with_equal_rule.py::test_should_be_sensitive_to_the_case PASSED [ 45%]
test_pivot_process_with_equal_rule.py::test_should_not_replace_a_word_not_in_the_list PASSED [ 50%]
test_pivot_process_with_equal_rule.py::test_should_not_replace_a_word_in_another_column PASSED [ 55%]
test_pivot_process_with_in_rule.py::test_should_replace_the_first_word_in_the_list PASSED [ 60%]
test_pivot_process_with_in_rule.py::test_should_replace_the_second_word_in_the_list PASSED [ 65%]
test_pivot_process_with_in_rule.py::test_should_not_be_sensitive_to_the_case PASSED [ 70%]
test_pivot_process_with_in_rule.py::test_should_not_replace_a_word_not_in_the_list PASSED [ 75%]
test_pivot_process_with_in_rule.py::test_should_not_replace_a_word_in_another_column PASSED [ 80%]
test_pivot_process_with_substr_rule.py::test_should_work_for_the_first_substring_in_the_list PASSED [ 85%]
test_pivot_process_with_substr_rule.py::test_should_not_work_if_not_in_the_list PASSED [ 90%]
test_pivot_process_with_substr_rule.py::test_should_replace_the_fourth_word_in_the_list PASSED [ 95%]
test_pivot_process_with_substr_rule.py::test_should_not_be_sensitive_to_the_case PASSED [100%]
=============================== warnings summary ===============================
../filetransformer/tools_bypass.py:275
../filetransformer/tools_bypass.py:275: DeprecationWarning: invalid escape sequence \[
"""
../filetransformer/tools_bypass.py:605
../filetransformer/tools_bypass.py:605: DeprecationWarning: invalid escape sequence \(
m = re.match('REPLACE\({?(?P<column_name>\w+)}?, "*(?P<value_to_replace>\w*)"*, "*(?P<value_output>\w*)"*\)', output_value)
test_pivot_process_with_equal_rule.py::test_should_replace_the_first_word_in_the_list
test_pivot_process_with_equal_rule.py::test_should_replace_the_second_word_in_the_list
test_pivot_process_with_equal_rule.py::test_should_be_sensitive_to_the_case
test_pivot_process_with_equal_rule.py::test_should_not_replace_a_word_not_in_the_list
test_pivot_process_with_in_rule.py::test_should_replace_the_first_word_in_the_list
test_pivot_process_with_in_rule.py::test_should_replace_the_second_word_in_the_list
test_pivot_process_with_in_rule.py::test_should_not_be_sensitive_to_the_case
test_pivot_process_with_in_rule.py::test_should_not_replace_a_word_not_in_the_list
/usr/local/lib/python3.6/site-packages/pandas/core/indexing.py:205: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
self._setitem_with_indexer(indexer, value)
-- Docs: https://docs.pytest.org/en/stable/warnings.html
- generated xml file: /home/********/workspace/src/tests/test-results/coverage.xml -
----------- coverage: platform linux, python 3.6.8-final-0 -----------
Coverage HTML written to dir test-results/cov_html
Coverage XML written to file test-results/results.xml
======================= 20 passed, 10 warnings in 1.07s ========================
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
/home/********/workspace/src/tests
-rwxrwxrwx 1 ******** ******** 3894 Aug 5 13:16 test-results/coverage.xml
-rwxrwxrwx 1 ******** ******** 35873 Aug 5 13:16 test-results/results.xml
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$#
total 96
drwxr-xr-x 6 ******** ******** 4096 Aug 5 13:16 .
drwxr-xr-x 14 ******** ******** 4096 Aug 5 13:15 ..
-rw-r--r-- 1 ******** ******** 53248 Aug 5 13:16 .coverage
drwxr-xr-x 3 ******** ******** 4096 Aug 5 13:16 .pytest_cache
-rw-r--r-- 1 ******** ******** 0 Aug 5 13:15 __init__.py
drwxr-xr-x 2 ******** ******** 4096 Aug 5 13:16 __pycache__
drwxr-xr-x 3 ******** ******** 4096 Aug 5 13:16 test-results
-rw-r--r-- 1 ******** ******** 1437 Aug 5 13:15 test_parse_params_for_replace.py
-rw-r--r-- 1 ******** ******** 1417 Aug 5 13:15 test_pivot_process_with_date_rule.py
-rw-r--r-- 1 ******** ******** 2690 Aug 5 13:15 test_pivot_process_with_equal_rule.py
-rw-r--r-- 1 ******** ******** 2684 Aug 5 13:15 test_pivot_process_with_in_rule.py
-rw-r--r-- 1 ******** ******** 3450 Aug 5 13:15 test_pivot_process_with_substr_rule.py
drwxr-xr-x 5 ******** ******** 4096 Aug 5 13:15 tmp
############################
Feature: in operator
Background:
Scenario: the first word in the list should be replaced
Given a rule
Given a rule
And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
And this rule outputs on the column "first_input_column" with the value "'mot'"
And this rule outputs on the column "first_input_column" with the value "'mot'"
Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
When A file with "Mots" in the "first_input_column" is transformed
When A file with "Mots" in the "first_input_column" is transformed
Then I get an output file with "mot" in the "first_pivot_column"
Then I get an output file with "mot" in the "first_pivot_column"
Scenario: the second word in the list should be replaced
Given a rule
Given a rule
And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
And this rule outputs on the column "first_input_column" with the value "'mot'"
And this rule outputs on the column "first_input_column" with the value "'mot'"
Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
When A file with "OtherMot" in the "first_input_column" is transformed
When A file with "OtherMot" in the "first_input_column" is transformed
Then I get an output file with "mot" in the "first_pivot_column"
Then I get an output file with "mot" in the "first_pivot_column"
Scenario: a complete test
Given a rule
Given a rule
And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
And this rule has a first operator "in" set on the column "first_input_column" with the value "['Mots', 'OtherMot']"
And this rule outputs on the column "first_input_column" with the value "'mot'"
And this rule outputs on the column "first_input_column" with the value "'mot'"
Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
Given the input columns "first_input_column" associated to the pivot columns "first_pivot_column"
Given the input columns "first_input_column/second_input_column" associated to the pivot columns "first_pivot_column/second_pivot_column"
Given the input columns "first_input_column/second_input_column" associated to the pivot columns "first_pivot_column/second_pivot_column"
When the following file is transformed
| first_input_column | second_input_column |
| Mots | * |
| OtherMot | * |
| othermot | * |
| word | * |
| * | OtherMot |
When the following file is transformed
| first_input_column | second_input_column |
| Mots | * |
| OtherMot | * |
| othermot | * |
| word | * |
| * | OtherMot |
Then I get the following file
| first_pivot_column | second_pivot_column |
| mot | * |
| mot | * |
| mot | * |
| word | * |
| * | OtherMot |
Then I get the following file
| first_pivot_column | second_pivot_column |
| mot | * |
| mot | * |
| mot | * |
| word | * |
| * | OtherMot |
1 feature passed, 0 failed, 0 skipped
3 scenarios passed, 0 failed, 0 skipped
19 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.033s