Submodule scan in Bitbucket pipelines

MonicaS · October 7, 2020, 1:32pm

Hi,

I am trying to set up SonarCloud scanning for C# submodules.
Here is my bitbucket pipeline.yml config

> image: atlassian/default-image:2
> 
> clone:
>       depth: full            
> 
> definitions:
>       caches:
>             sonar: ~/.sonar/cache 
>       steps:
>       - step: &build-test-sonarcloud
>               name: Build, test and analyze on SonarCloud
>               caches:
>               - sonar
>               script:
>               - git submodule update --recursive               
>               - pipe: sonarsource/sonarcloud-scan:1.2.1

>       - step: &check-quality-gate-sonarcloud
>               name: Check the Quality Gate on SonarCloud
>               script:
>               - pipe: sonarsource/sonarcloud-quality-gate:0.1.3
> 
> pipelines:                
>   branches:
>     master:
>       - step: *build-test-sonarcloud
>       - step: *check-quality-gate-sonarcloud
>   pull-requests:
>     '**':
>       - step: *build-test-sonarcloud
>       - step: *check-quality-gate-sonarcloud

The first part build-test-sonarcloud passes but then I get an error when trying to check the quality gate

Status: Downloaded newer image for sonarsource/sonarcloud-quality-gate:0.1.3

✖ Could not check Quality Gate status

I just started trying out SonarCloud so I am not sure what I am missing. I got it to work just fine with a normal C# repository and data/reports were uploaded just fine.

Thanks in advance!

Claire_Villard · October 8, 2020, 4:27pm

Hi Monica, and welcome to our community forum!

Have you checked the Quality Gate status of your project on SonarCloud? Is it possible that it has not been computed, because it’s the first scan of your project, or because you have not set the New Code period setting and are using the default “Sonar Way” quality gate?

If it is the case, I invite you to follow the New Code Definition documentation, to set up the period SonarCloud uses to check the “New Code” conditions of the Quality Gate for your project.
The setting will take effect during the next scan of your project, and the pipeline should be OK if it passes.

MonicaS · October 9, 2020, 8:13am

Hi Claire,

I just tried out with a different custom quality gate, and the build passed. However, my code does seem to be uploaded/analyzed in the portal. I believe this has to do with the submodule?

Claire_Villard · October 22, 2020, 10:09am

Hi Monica,

Is the issue fixed?
If not, we released a new version of the sonarcloud-quality-gate pipe, version 0.1.4, which bring support to “not computed” quality gates recently introduced in SonarCloud.

Could you please clarify what you mean by “my code does seem to be uploaded/analyzed in the portal”, and the problem with submodule?

MonicaS · October 22, 2020, 12:32pm

Hi Claire,

The issue around the “Could not check quality gate” was fixed by changing the Quality Gate itself. I did work with both .14 and .13.
However, the code does not seem to be scanned. It does say scanning complete in bitbucket but in the SonarCloud portal, the repository is empty and no lines of code were analyzed.

Claire_Villard · October 22, 2020, 12:53pm

Do you have some special configuration for the sonarsource/sonarcloud-scan pipe, or on SonarCloud side (Project administration > General Settings > Analysis Scope), that could exclude the source files from the analysis (here is the documentation about analysis scope configuration)?

Could you please post the logs of the sonarsource/sonarcloud-scan pipe execution?

Also, are there analysis warnings displayed at the top right corner of the SonarCloud project page?

Capture d’écran de 2020-10-22 14-51-07

MonicaS · October 22, 2020, 1:08pm

There is nothing special set on either side. I only wanted a dummy scan to make sure the subrepo can be analyzed. No warnings are displayed

Here are the logs of the pipe execution.

INFO: User cache: /root/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=134ms
INFO: Load/download plugins (done) | time=28758ms
INFO: Loaded core extensions: developer-scanner
INFO: JavaScript/TypeScript frontend is enabled
INFO: Detected project key 'mstoicayellowtail_submodulepoc' from 'Bitbucket Cloud Pipelines'
INFO: Detected organization key 'mstoicayellowtail' from 'Bitbucket Cloud Pipelines'
INFO: Process project properties
INFO: Execute project builders
INFO: Execute project builders (done) | time=1ms
INFO: Project key: mstoicayellowtail_submodulepoc
INFO: Base dir: /opt/atlassian/pipelines/agent/build
INFO: Working dir: /opt/atlassian/pipelines/agent/build/.scannerwork
INFO: Load project settings for component key: 'mstoicayellowtail_submodulepoc'
INFO: Load project settings for component key: 'mstoicayellowtail_submodulepoc' (done) | time=224ms
INFO: Found an active CI vendor: 'Bitbucket Pipelines'
INFO: Load project branches
INFO: Load project branches (done) | time=110ms
INFO: Check ALM binding of project 'mstoicayellowtail_submodulepoc'
INFO: Detected project binding: BOUND
INFO: Check ALM binding of project 'mstoicayellowtail_submodulepoc' (done) | time=102ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=101ms
INFO: Load branch configuration
INFO: Detected analysis for branch 'master'
INFO: Auto-configuring branch master
INFO: Load branch configuration (done) | time=3ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=137ms
INFO: Load active rules
INFO: Load active rules (done) | time=3655ms
INFO: Organization key: mstoicayellowtail
INFO: Branch name: master, type: long living
INFO: Indexing files...
INFO: Project configuration:
INFO: 2 files indexed
INFO: ------------- Run sensors on module mstoicayellowtail_submodulepoc
INFO: JavaScript/TypeScript frontend is enabled
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=129ms
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$1 (file:/root/.sonar/cache/a89f1943fc75b65becd9fb4ecab8d913/sonar-tsql-plugin.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: Sensor CSS Rules [cssfamily]
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [cssfamily] (done) | time=9ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=1ms
INFO: Sensor JavaXmlSensor [java]
INFO: Sensor JavaXmlSensor [java] (done) | time=1ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=4ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
INFO: Sensor JavaSecuritySensor [security]
INFO: Reading type hierarchy from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/java
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/java
INFO: No UCFGs have been included for analysis.
INFO: Sensor JavaSecuritySensor [security] (done) | time=3ms
INFO: Sensor CSharpSecuritySensor [security]
INFO: Reading type hierarchy from: /opt/atlassian/pipelines/agent/build/ucfg_cs2
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/ucfg_cs2
INFO: No UCFGs have been included for analysis.
INFO: Sensor CSharpSecuritySensor [security] (done) | time=0ms
INFO: Sensor PhpSecuritySensor [security]
INFO: Reading type hierarchy from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/php
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/php
INFO: No UCFGs have been included for analysis.
INFO: Sensor PhpSecuritySensor [security] (done) | time=1ms
INFO: Sensor PythonSecuritySensor [security]
INFO: Reading type hierarchy from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/python
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/python
INFO: No UCFGs have been included for analysis.
INFO: Sensor PythonSecuritySensor [security] (done) | time=1ms
INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /opt/atlassian/pipelines/agent/build/.scannerwork/ucfg2/js
INFO: No UCFGs have been included for analysis.
INFO: Sensor JsSecuritySensor [security] (done) | time=0ms
INFO: ------------- Run sensors on project
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=1ms
INFO: SCM Publisher is disabled
INFO: CPD Executor Calculating CPD for 0 files
INFO: CPD Executor CPD calculation finished (done) | time=0ms
INFO: Analysis report generated in 268ms, dir size=156 KB
INFO: Analysis report compressed in 14ms, zip size=29 KB
INFO: Analysis report uploaded in 245ms
INFO: ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=mstoicayellowtail_submodulepoc&branch=master
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=AXUINqh7ZsC6lpQfjmAR
INFO: Analysis total time: 9.183 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 42.196s
INFO: Final Memory: 9M/40M
INFO: ------------------------------------------------------------------------
✔ SonarCloud analysis was successful.

Claire_Villard · October 22, 2020, 2:54pm

Looking at the logs and at your demo project, it seems to me that the submodule is not cloned into its directory.

I tried locally, and after calling git submodule update --recursive --depth 100 the submodule directory was still empty.
But if I add this command before : git submodule init, then the update command actually clones the content into the directory.

Maybe updating your pipeline to add the git submodule init command before the git submodule update could solve?

MonicaS · October 22, 2020, 2:30pm

Hi Claire, I added the following command but nothing changed. The build results are the same and the code/scan results are still not in the portal sadly.
I can now see this, but still, no lines analyzed.
Screenshot 2020-10-22 at 16.29.40

Claire_Villard · October 22, 2020, 2:38pm

Hi,

I see an improvement : the code is shown in the “code” tab, while I think it was empty before.
So to me, the submodule issue is fixed.

For the measures and quality gate being empty, I see your code is C#.
C# code can’t be analyzed directly like this. It needs a special scanner, called Scanner for MSBuild.

From the scanner logs:

WARN: Your project contains C# files which cannot be analyzed with the scanner you are using. To analyze C# or VB.NET, you must use the Scanner for MSBuild 4.x

The documentation to configure the Scanner for MSBuild is here.

MonicaS · October 22, 2020, 2:48pm

Hi Claire,

Thanks for clarifying this. I found some more documentation on the subject.
Thanks for your help! I guess the ticket can now be closed

Claire_Villard · October 22, 2020, 2:54pm