Sonarcloud code coverage of C++ in GitHub Actions

acgetchell · July 10, 2021, 4:51pm

Using GitHub Actions, I can get build-wrapper and sonar-scanner to analyze my C++ code.

acgetchell/CDT-plusplus/blob/develop/.github/workflows/main.yml#L68

    
      
                echo $HOME
          
          
  - name: Build
              shell: bash 
              run: |
                pwd
                vcpkg install --feature-flags=manifests
                export VCPKG_ROOT="$VCPKG_INSTALLATION_ROOT"
                cmake -G Ninja -D CMAKE_BUILD_TYPE=Debug -S . -B build
                cmake --build build
          
          
sonarcloud:
            name: SonarCloud
            runs-on: ubuntu-latest
            env:
              SONAR_SCANNER_VERSION: 4.6.1.2450 # Find the latest version in the "Linux" link on this page:
                                                # https://sonarcloud.io/documentation/analysis/scan/sonarscanner/
              SONAR_SERVER_URL: "https://sonarcloud.io"
              BUILD_WRAPPER_OUT_DIR: build_wrapper_output_directory # Directory where build-wrapper output will be placed
            steps:
              - uses: actions/checkout@v2

What doesn’t work, so far, is obtaining Code Coverage.

github.com

acgetchell/CDT-plusplus/blob/develop/.github/workflows/main.yml#L148

    
      
              curl -sSLo $HOME/.sonar/build-wrapper-linux-x86.zip ${{ env.BUILD_WRAPPER_DOWNLOAD_URL }}
              unzip -o $HOME/.sonar/build-wrapper-linux-x86.zip -d $HOME/.sonar/
          
          
- name: Set build-wrapper and sonar-scanner paths
            run: |
              echo "$HOME/.sonar/build-wrapper-linux-x86" >> $GITHUB_PATH
              echo "$HOME/.sonar/sonar-scanner-${{ env.SONAR_SCANNER_VERSION }}-linux/bin" >> $GITHUB_PATH
          
          
- name: Build
            run: build-wrapper-linux-x86-64 --out-dir ${{ env.BUILD_WRAPPER_OUT_DIR }} cmake --build build
          
          
- name: Test and generate coverage
            continue-on-error: true
            run: |
              cd $GITHUB_WORKSPACE/build
              ctest ctest --schedule-random -j2
          
          

          
- name: Compile coverage reports
            run: |
              cd $GITHUB_WORKSPACE/build

I’m using lcov in the same way I use it on Travis-CI to generate coverage reports in CodeCov:

github.com

acgetchell/CDT-plusplus/blob/develop/.travis.yml#L141

    
      
          - os: linux
            compiler: clang-10
            env: TSAN=true
            before_script:
              - export CMAKE_ARGS="-D ENABLE_SANITIZER_THREAD:BOOL=TRUE"
            after_success:
              - cd $TRAVIS_BUILD_DIR/build/
              - travis_wait 90 ctest -VV
              - cd src
              - ./initialize --s -n32000 -t11 -o
          # CodeCov
          - os: linux
            compiler: g++-10
            env: CODECOV=true
            before_script:
              - export CMAKE_ARGS="-D ENABLE_COVERAGE:BOOL=TRUE"
            script:
              - mkdir build
              - cd build || exit
              - cmake $CMAKE_ARGS ..
              - cmake --build . --config Debug

https://app.codecov.io/gh/acgetchell/CDT-plusplus/

That is, essentially:

cmake -G Ninja -D CMAKE_BUILD_TYPE=Debut -D ENABLE_COVERAGE:BOOL=TRUE -S . -B build
build-wrapper-linux-x86-64 --out-dir ${{ env.BUILD_WRAPPER_OUT_DIR }} cmake --build build
cd $GITHUB_WORKSPACE/build
ctest ctest --schedule-random -j2
lcov --capture --directory . --output-file coverage.info
lcov --remove coverage.info '/usr/*' '*/usr/include/*' '*/vcpkg_installed/*' --output-file coverage.info

(The test step is set to continue-on-error, because some random number tests will fail and I don’t want that to abort the run.)

I used sonar-project.properties to specify the location of the lcov report:

github.com

acgetchell/CDT-plusplus/blob/develop/sonar-project.properties#L21

    
      
          #sonar.projectName=CDT-plusplus
          #sonar.projectVersion=1.0
          
          
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
          sonar.sources=src,include
          sonar.tests=tests
          sonar.cfamily.build-wrapper-output=build_wrapper_output_directory
          sonar.cfamily.cache.enabled=true
          sonar.cfamily.cache.path=.sonar
          sonar.cfamily.threads=2
          sonar.cfamily.llvm-cov.reportPath=build/coverage.info
          
          
# Encoding of the source code. Default is default system encoding
          #sonar.sourceEncoding=UTF-8

If I have to use gcov instead, is there a way to specify the location of all the gcda files, similar to what lcov does?

adam@hapkido ~/projects/CDT-plusplus/build (develop*) $ lcov --capture --directory . --output-file coverage.info
Capturing coverage data from .
Found LLVM gcov version 12.0.5, which emulates gcov version 4.2.0
Using intermediate gcov format
Scanning . for .gcda files ...
Found 21 data files in .
Processing tests/CMakeFiles/CDT_test.dir/Move_always_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/main.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Foliated_triangulation_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Move_tracker_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Apply_move_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Ergodic_moves_3_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Metropolis_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Settings_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Move_command_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Function_ref_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/S3Action_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Tetrahedron_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Geometry_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Torus_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Sphere_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Manifold_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Vertex_test.cpp.gcda
Processing tests/CMakeFiles/CDT_test.dir/Utilities_test.cpp.gcda
Processing src/CMakeFiles/initialize.dir/initialize.cpp.gcda
Processing src/CMakeFiles/cdt-opt.dir/cdt-opt.cpp.gcda
Processing src/CMakeFiles/cdt-gv.dir/cdt-gv.cpp.gcda
Finished .info-file creation

The remaining steps on lcov are useful for filtering out coverage on included libraries, I don’t know how to do that with gcov here.

Thanks for any suggestions.

mpaladin · July 15, 2021, 7:32am

Hi @acgetchell ,

this is what we do to import gcov reports for our project:

mkdir ${GCOV_DIR_NAME}
pushd ${GCOV_DIR_NAME}
for f in `find ../build/cmake/CMakeFiles/core.dir -name '*.o'`; do
  echo "Processing $f file..."
  gcov -o ${f} x
done
ls | wc -l
popd

and then we pass -Dsonar.cfamily.gcov.reportsPath=${GCOV_DIR_NAME} to sonar-scanner.

Another option, which I haven’t tried myself but should work, is to use gcovr using --sonarqube option and then passing the output result to sonar.coverageReportPaths property.

acgetchell · July 16, 2021, 7:51pm

Hi @mpaladin,

I tried the method suggested above.

github.com

acgetchell/CDT-plusplus/blob/develop/.github/workflows/main.yml#L158

    
      
          - name: Build
            run: build-wrapper-linux-x86-64 --out-dir ${{ env.BUILD_WRAPPER_OUT_DIR }} cmake --build build
          
          
- name: Test and generate coverage
            continue-on-error: true
            run: |
              cd $GITHUB_WORKSPACE/build
              ctest --schedule-random -j2
          
          

          
- name: Compile coverage reports
            run: |
              cd $GITHUB_WORKSPACE/build
              mkdir gcov-reports
              pushd gcov-reports
              for f in `find ../tests/CMakeFiles/CDT_test.dir -name '*.o'`; do
                echo "Processing $f file..."
                gcov -o ${f} x
              done
              ls | wc -l
              popd

The problem is that it generates coverage on 226 files, most of which are libraries installed via vcpkg. There are only 21 relevant files, and my coverage reports are dramatically affected including everything.

lcov solves that problem by being able to filter extensively.

For whatever reason, it did not upload to SonarCloud.

https://sonarcloud.io/summary/new_code?id=acgetchell_CDT-plusplus

I believe I set the correct path for the reports:

github.com

acgetchell/CDT-plusplus/blob/develop/sonar-project.properties#L21

    
      
          #sonar.projectName=CDT-plusplus
          #sonar.projectVersion=1.0
          
          
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
          sonar.sources=src,include
          sonar.tests=tests
          sonar.cfamily.build-wrapper-output=build_wrapper_output_directory
          sonar.cfamily.cache.enabled=true
          sonar.cfamily.cache.path=.sonar
          sonar.cfamily.threads=2
          sonar.cfamily.gcov.reportPath=build/gcov-reports
          
          
# Encoding of the source code. Default is default system encoding
          #sonar.sourceEncoding=UTF-8

Amelie · July 19, 2021, 2:36pm

Hello @acgetchell ,

The reason it did not upload to SonarCloud may be that there is a typo in sonar-project.properties: it should be sonar.cfamily.gcov.reportsPath, with an “s” in “reports”.

(About the filtering, I don’t know gcov well enough to guide you, but it may be easier by using gcovr and the sonar.coverageReportPaths property that Massimo mentioned)

acgetchell · July 23, 2021, 1:51am

Thank you @Amelie! That indeed resolved the issue.

system · July 30, 2021, 1:52am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

mpaladin · August 2, 2021, 12:33pm

Hi @acgetchell,

what do you mean with dramatically affected?