Sonarqube not scanning the Whole Repo

SonarQube Server / Community Build
, ,
1

I have Terraform codes on the AWS CodeCommit repo inside the two major folders that is “_locals” and “environments”, but when I scan the whole repo, only the “_locals” and its subfolders and files are scanned, the second major folder “environment” is excluded.

I am using the Sonarqube Community Edition - Version 10.4.1 (build 88267) integrated with Jenkins, and the following is the Jenkins script that I use:

sonar.projectKey=Know-itall-TerraformCode-Analysis
sonar.language=terraform
sonar.sources=.
sonar.terraform.binaries=.
sonar.exclusions=**/*.py,**/*.yml,**/*.yaml

#This is the AWS CodeCommit Repo structure:

codecommit
codecommit822×422 13.9 KB

#This is the SonarQube Scanned code output (it only scans the _locals folder and not scan other folder “environments” inside the same repo)

sonarqube_scanned_code
sonarqube_scanned_code1861×652 37.4 KB

#This is the Jenkins Build Output when the Repo is scanned.

Started by user Desh
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/Know-itall-TerraformCode
The recommended git tool is: NONE
using credential CodeCommitCred
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Know-itall-TerraformCode/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://git-codecommit.us-east-1.amazonaws.com/v1/repos/Know-itall-IAC # timeout=10
Fetching upstream changes from https://git-codecommit.us-east-1.amazonaws.com/v1/repos/Know-itall-IAC
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
using GIT_ASKPASS to set credentials 
 > git fetch --tags --force --progress -- https://git-codecommit.us-east-1.amazonaws.com/v1/repos/Know-itall-IAC +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 7e02dc194219db6529c98c485d5f0ea7233fedb1 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 7e02dc194219db6529c98c485d5f0ea7233fedb1 # timeout=10
Commit message: "IaC code to controltower"
 > git rev-list --no-walk 7e02dc194219db6529c98c485d5f0ea7233fedb1 # timeout=10
[Know-itall-TerraformCode] $ /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarServer/bin/sonar-scanner -Dsonar.host.url=http://52.200.39.130:9000/ ******** -Dsonar.projectKey=Know-itall-TerraformCode-Analysis -Dsonar.language=terraform -Dsonar.sources=. -Dsonar.terraform.binaries=. -Dsonar.exclusions=**/*.py,**/*.yml,**/*.yaml -Dsonar.projectBaseDir=/var/lib/jenkins/workspace/Know-itall-TerraformCode
INFO: Scanner configuration file: /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarServer/conf/sonar-scanner.properties
INFO: Project root configuration file: NONE
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.10 Private Build (64-bit)
INFO: Linux 5.15.0-1056-aws amd64
INFO: User cache: /var/lib/jenkins/.sonar/cache
INFO: Analyzing on SonarQube server 10.4.1.88267
INFO: Default locale: "en", source code encoding: "UTF-8" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=111ms
INFO: Server id: D4088475-AY5hm8q0VgPdKZXJ0gYh
INFO: User cache: /var/lib/jenkins/.sonar/cache
WARN: sonar.plugins.downloadOnlyRequired is false, so ALL available plugins will be downloaded
INFO: Loading all plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=58ms
INFO: Load/download plugins
INFO: Load/download plugins (done) | time=174ms
INFO: Process project properties
INFO: Process project properties (done) | time=12ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=2ms
INFO: Project key: Know-itall-TerraformCode-Analysis
INFO: Base dir: /var/lib/jenkins/workspace/Know-itall-TerraformCode
INFO: Working dir: /var/lib/jenkins/workspace/Know-itall-TerraformCode/.scannerwork
INFO: Load project settings for component key: 'Know-itall-TerraformCode-Analysis'
INFO: Load project settings for component key: 'Know-itall-TerraformCode-Analysis' (done) | time=31ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=112ms
INFO: Auto-configuring with CI 'Jenkins'
INFO: Load active rules
INFO: Load active rules (done) | time=2894ms
INFO: Load analysis cache
INFO: Load analysis cache (404) | time=10ms
INFO: Preprocessing files...
INFO: 1 language detected in 51 preprocessed files
INFO: 2 files ignored because of inclusion/exclusion patterns
INFO: 0 files ignored because of scm ignore settings
INFO: Load project repositories
INFO: Load project repositories (done) | time=21ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Excluded sources: **/*.py, **/*.yml, **/*.yaml
INFO: 51 files indexed
INFO: Quality profile for terraform: Sonar way
INFO: ------------- Run sensors on module Know-itall-TerraformCode-Analysis
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=44ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3ms
INFO: Sensor IaC Terraform Sensor [iac]
INFO: 13 source files to be analyzed
INFO: 13/13 source files have been analyzed
INFO: Sensor IaC Terraform Sensor [iac] (done) | time=1090ms
INFO: Sensor CSS Rules [javascript]
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=1ms
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=2ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Sensor C# Analysis Log [csharp] (done) | time=20ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=4ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: 13 source files to be analyzed
INFO: 13/13 source files have been analyzed
INFO: Sensor TextAndSecretsSensor [text] (done) | time=1799ms
INFO: Sensor VB.NET Project Type Information [vbnet]
INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=2ms
INFO: Sensor VB.NET Analysis Log [vbnet]
INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=22ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=93ms
INFO: ------------- Run sensors on project
INFO: Sensor Analysis Warnings import [csharp]
INFO: Sensor Analysis Warnings import [csharp] (done) | time=1ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=2ms
INFO: SCM Publisher SCM provider for this project is: git
INFO: SCM Publisher 13 source files to be analyzed
INFO: SCM Publisher 13/13 source files have been analyzed (done) | time=510ms
INFO: CPD Executor Calculating CPD for 0 files
INFO: CPD Executor CPD calculation finished (done) | time=0ms
INFO: Analysis report generated in 184ms, dir size=292.8 kB
INFO: Analysis report compressed in 68ms, zip size=61.6 kB
INFO: Analysis report uploaded in 32ms
INFO: ANALYSIS SUCCESSFUL, you can find the results at: http://52.200.39.130:9000/dashboard?id=Know-itall-TerraformCode-Analysis
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://52.200.39.130:9000/api/ce/task?id=b005417f-00ab-4c06-b75a-4603f4a247a6
INFO: Analysis total time: 12.292 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 14.247s
INFO: Final Memory: 21M/80M
INFO: ------------------------------------------------------------------------
Finished: SUCCESS

Please help me find what is causing this.

3

While further analyzing, I found that the files inside the “environment” folder had the file named “terragrunt.hcl” and this may be why Sonarqube could not read it.

Please help me know if it’s possible and worth it to scan these files and if I will even get the scanned results on SonarQube for these files.

5

Hi @desh,

Thank you for clarifying your use case, even though we did not respond yet. By default, the TerraformSensor picks up .tf files. To analyze .hcl files, you could add the file suffix to the sonar.terraform.file.suffixes property. Nevertheless, even when our parser can parse HCL as syntax, I don’t know if we provide any rules for the content. Can you elaborate on what issues you expect to see in these files?

Side question: You define the sonar.terraform.binaries. Where is this property documented?

Best,

1 Like
6

Hello @Nils_Werner,

Thank you for your response.

Ok, so once I define the .hcl files in the suffix like this.

Will it treat them as terraform code only or will scan them under some other programming language?

Side question: You define the sonar.terraform.binaries . Where is this property documented? → I found this in one of the online documentation. Once I skip this “sonar.terraform.binaries=.”, the scan still happens.
Also can you a bit define when to use ‘sonar.terraform.binaries’ and is this mandatory to use.

7

Hi @desh,

Only the TerraformSensor can parse and analyze HCL syntax. So, it will be handled as Terraform code. Can you clarify what the HCL files in the environments directory are used for?

Regarding the sonar.terraform.binaries property. I wondered where this property is documented, as it is not a known property of the analyzer. I think it can come from a third-party plugin.

Best,