The project is an embedded application which is built using Xilinx’s PetaLinux tool suite. Under the hood, PetaLinux just uses bitbake. The build runs on an x86_64 machine (hence the usage of the x86_64 wrapper) and uses an aarch64 cross compiler. The application itself resides in a project tree which is controlled by git. When bitbake performs the build, it copies all of the application source files into a temporary directory. The build itself is deep within the layered structure of bitbake and AFAIK cannot be changed. The application source and temporary build path used by bitbake, respectively, are as follows:
I have the whole pipeline working. That is, the wrapper is able to generate build-wrapper-dump.json and build-wrapper.log and the scanner successfully ingests these files. The scanner then posts the results to SonarCloud.
This is where I run into an issue. Since the build wrapper is prepended to the compile command, it runs in the temporary directory created by bitbake. The scanner runs immediately after the application is built and is also executed in that temporary directory. The temporary build directory is explicitly ignored in .gitignore file of the project. The scanner then produces the following message(s):
...
23:31:11.387 DEBUG: File '/home/<user>/<path_to_project>/build/tmp/work/aarch64-xilinx-linux/<application>/1.0-r0/<file>.cpp' is excluded by the scm ignore settings.
...
This makes sense because .gitignore explicitly sets build/ as one of the items to be ignored. I temporarily commented out the build/ entry and the scanner did seem to work. However, all scores were As. I also tried running with -Dsonar.scm.exclusions.disabled=true but that results in behavior identical to the previous method.
Is it possible to use the scanner in the above setting? I appreciate any help.
When you say “All scores were As” – are you referring to the New Code or Overall Code (or both?). I can’t help but wonder if your scan results are “hidden” behind Overall Code, since you had run some scans before you disabled SCM-based exclusions.
A screenshot of what you’re seeing SonarCloud-side is always helpful.
You are making an interesting point. I am in the process of implementing the SonarCloud pipeline. In this case I think, “All scores…” would mean Overall Code.
But you are correct. I have been running test builds all along. During each run, I change a dummy entry in a header file to force bitbake to perform a build. The entry is simply a #define DUMMY <n>, where <n> is just a number which changes from build to build. This value is not used anywhere but I don’t think this would cause any issues to be detected by the scanner. Maybe I am wrong though? That being said, the build of the application produces warnings anyway which, I think, the scanner would definitely catch.
One detail which I didn’t mention, when I make the change in between test runs, I am not performing any commits, pushes or merges. All changes are local to my copy of the repo. Maybe that is the cause of the problem?
Each new scan is noted by the Last analysis… field so I know that the scan runs and the whole pipeline is executed (this last analysis is a bit old because I haven’t run any tests since then). However, the commit number never changes. Below is the screen capture of the Last analysis… and the SonarCloud web page.
I noticed that the scanner generates a cache during each build. If I am not mistaken, the cache resides in the ~/.sonar/ directory. Would it be safe to delete it? Maybe this is why the scanner is not “seeing” any changes.
You are definitely performing an analysis of a pull request / short-lived branch (this is very crucial as that means only changed lines are accounted for, rather than your project overall). While your entire codebase will be analyzed scanner-side, the results will be filtered to changed code for the view in SonarCloud.
And, in order for those lines to have been detected as changed, you must have committed them to your repository. This is where some folks stumble when testing things locally, but doesn’t reallycome up once this is integrated into your CI pipeline.
When scanning your main branch / a long-lived branch, the full results will be shown (there will be a tab for Overall Code which shows results for your entire codebase)
So this understanding (very good instinct!) is correct
Just a quick update. A commit and a push did, indeed, result in a run of the scanner. However, because the build was pointing to a temporary sandbox prepared by bitbake, it included a lot more files in addition to sources of the application. As a result, SonarCloud hit the maximum number of lines which it is configured to examine (about 1 million) and the scan failed.
I was hoping that settings specified in .gitignore could be overridden using the inclusions parameter but that did not work. That is, if the sandbox created by bitbake is set to be ignored in .gitignore, it seems that it is not possible to tell the scanner to override this setting just for the set of source files which make up the application (but still reside in the sandbox).
I am not sure if there is a way to accomplish this but if there is, I would appreciate a hint on how it could be done.
My understanding is that you currently have -Dsonar.scm.exclusions.disabled=true set, is that correct? In which case the scanner is completely ignoring the .gitignore file.
So you have three options if you keep -Dsonar.scm.exclusions.disabled=true set
set sonar.sources to define exactly what directory contains your source code
Set sonar.exclusions to exclude files that are not the source of your application
Set sonar.inclusions to only include the source files of your application
thanks for explaining this. Just make sure I understand. Will the method you propose work even if application sources are in the sandbox directory created by bitbake? In other words if, say, the sandbox at <project dir>/build/ is ignored within .gitignore but contains application sources, will using the command set you show above instruct the scanner to ignore all files under <project dir>/build except application sources?
I’m assuming you will have to keep -Dsonar.scm.exclusions.disabled=true set due to the nature of your build, so we can ignore the existence of .gitignore entirely.
Using either sonar.exclusions or sonar.inclusions will allow you to either whittle away what’s in that directory that will be analyzed by the scanner, or define exactly what is analyzed. I think the docucmentation on Analysis Scope is helpful here.
Imagine that you are running the scanner from your project directory, and you have a /src/ folder within your build directory that has all the files you actually want to analyze.
You could set:
sonar.sources=build/src/
or sonar.inclusions=build/src/**/*
or sonar.exclusions=<some glob pattern that identifies the non-source files>
