Hi all,
When I scan a short lived branch, I expect Sonar to show me only “new” issues introduced through the modified code.
But when I go to the web UI, I can see that old issues are also here:
I can of course filter by “creation date”, but I’m asking why a short lived scan which is supposed to scan only changes spots also old issues ?
Thanks
Hi,
Can you provide some details about your analysis? E.G. versions, analysis command line, …
Ann
Hi,
We scan the master branch once a week, then work on our project using git development branches.
The “version” has not changed (we do not use it for now).
Here is the output of the scan:
INFO: Running SonarQube scanner on the build
INFO: Scanner configuration file: /home/slankri/usr/sonar-scanner-3.2.0.1227-linux/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/slankri/work/DspRefBase-Hexagon/config/sonar/sonar-project.properties
INFO: SonarQube Scanner 3.2.0.1227
INFO: Java 1.8.0_121 Oracle Corporation (64-bit)
INFO: Linux 4.15.0-43-generic amd64
INFO: User cache: /home/slankri/.sonar/cache
INFO: SonarQube server 7.3.0
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Publish mode
INFO: Load global settings
INFO: Load global settings (done) | time=170ms
INFO: Server id: XXXXXXXXXXXXXX
INFO: User cache: /home/slankri/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=86ms
INFO: Load/download plugins (done) | time=212ms
INFO: Loaded core extensions: branch-scanner
INFO: Process project properties
INFO: Load project branches
INFO: Load project branches (done) | time=64ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=20ms
INFO: Load branch configuration
INFO: Load branch configuration (done) | time=3ms
INFO: Load project repositories
INFO: Load project repositories (done) | time=117ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=121ms
INFO: Load active rules
INFO: Load active rules (done) | time=1239ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=27ms
INFO: Project key: DspRefBase-Hexagon
INFO: Project base dir: /home/slankri/work/DspRefBase-Hexagon
INFO: Branch name: dev/QC-137, type: short living
INFO: ------------- Scan DspRefBase-Hexagon
INFO: Base dir: /home/slankri/work/DspRefBase-Hexagon
INFO: Working dir: /home/slankri/work/DspRefBase-Hexagon/build/Release/PREMIUM/sonar/scanner
INFO: Source paths: src
INFO: Source encoding: UTF-8, default locale: en_US
INFO: SCM collecting changed files in the branch
INFO: SCM collecting changed files in the branch (done) | time=10776ms
INFO: Load server rules
INFO: Load server rules (done) | time=227ms
INFO: Index files
INFO: Included sources:
INFO: **/*.cpp
INFO: **/*.c
INFO: **/*.h
INFO: Excluded sources:
INFO: **/test*/**
INFO: **/thirdparty/**/*
INFO: **/UserPresetConversion/**
INFO: **/audioengine/**/*
INFO: 239 files indexed
INFO: 1737 files ignored because of inclusion/exclusion patterns
INFO: Quality profile for c: Arkamys-MISRA
INFO: Quality profile for cpp: Arkamys-MISRA
INFO: Sensor SonarJavaXmlFileSensor [java]
INFO: Sensor SonarJavaXmlFileSensor [java] (done) | time=3ms
INFO: Sensor CFamily [cpp]
INFO: Using build-wrapper output: /home/slankri/work/DspRefBase-Hexagon/build/Release/PREMIUM/sonar/wrapper/build-wrapper-dump.json
INFO: Available processors: 8
INFO: Using 8 threads for analysis according to value of "sonar.cfamily.threads" property.
WARN: Metric 'comment_lines_data' is deprecated. Provided value is ignored.
WARN: Invalid character encountered in file /home/slankri/work/DspRefBase-Hexagon/src/audio/primitives/OverlapAdd512/OverlapAdd512_float.c at line 96 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
INFO: [pool-5-thread-1] ..........
INFO: [pool-5-thread-2] ......
INFO: 80 compilation units analyzed
INFO: Sensor CFamily [cpp] (done) | time=19485ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=55ms
INFO: Sensor JavaSecuritySensor [security]
INFO: UCFGs: 0, excluded: 0, source entrypoints: 0
INFO: No UCFGs have been included for analysis.
INFO: Sensor JavaSecuritySensor [security] (done) | time=3ms
INFO: Sensor CSharpSecuritySensor [security]
INFO: UCFGs: 0, excluded: 0, source entrypoints: 0
INFO: No UCFGs have been included for analysis.
INFO: Sensor CSharpSecuritySensor [security] (done) | time=0ms
INFO: SCM provider for this project is: git
INFO: 8 files to be analyzed
INFO: 8/8 files analyzed
INFO: Skipping CPD calculation for short living branch and pull request
INFO: Analysis report generated in 121ms, dir size=235 KB
INFO: Analysis reports compressed in 260ms, zip size=65 KB
INFO: Analysis report uploaded in 80ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://virt-win-ci:9000/dashboard?id=DspRefBase-Hexagon&branch=dev%2FQC-137&resolved=false
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://virt-win-ci:9000/api/ce/task?id=AWgyVMli6lClypMaviv-
INFO: Task total time: 36.213 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 37.888s
INFO: Final Memory: 27M/469M
INFO: ------------------------------------------------------------------------
INFO: Running SonarQube scanner on the build (done)
INFO: Waiting for Sonar Server to process the analysis: PENDING
INFO: Waiting for Sonar Server to process the analysis: IN_PROGRESS
INFO: Waiting for Sonar Server to process the analysis (done)
INFO: Downloading the analysis result
INFO: Downloading the analysis result (done)
INFO: ------------------------------------------------------------------------
WARN: /!\ Your code introduces 77 new issues:
WARN: MINOR=38, INFO=26, MAJOR=13, CRITICAL=0, BLOCKER=0
WARN: -> visit http://virt-win-ci:9000/dashboard?id=DspRefBase-Hexagon&branch=dev%2FQC-137&resolved=false for issues
INFO: ------------------------------------------------------------------------
Among the 77 issues spotted, 74 of them are not introduced by the changes in the devel branch, but already present in master branch.
(the aim of this devel branch was to fix bugs).
Hi,
By versions, I meant SonarQube, scanner, analyzers, …
Also, what’s your analysis command line, and what plugins (with versions, please) are in your instance. The Issue summary at the end of your analysis log is highly suspicious. IIRC that functionality went away quite a while ago.
Ann
We have:
As for the summary you are seeing, it is from our custom script that downloads the JSON report from Sonar and extracts info from it. We have the same number of issues when we go to the Web interface so I guess that is not a problem of us fetching the incorrect data.
Our analysis command line is very classic: Build wrapper of our build then the scanner.
roughly this is:
build-wrapper --out-dir=/path/to/build-dump our-build-script.sh
sonar-scanner -Dproject.settings=/path/to/sonar-project.properties -Dsonar.cfamily.build-wrapper-output=path/to/build-dump.json
Then we wait for the server to finish the analysis, download the JSON report and extract issue counts.
Thanks
Hi,
The fact that you’re specifying the path to sonar-project.properties indicates to me that it’s not in the directory you’re analyzing from. So I wonder if this is relevant:
Ann
Thanks.
It is true that we are not analyzing from the “source” directory.
Here is our project layout (Git SCM):
# git root (TOP_DIR)
project/
# all sources
project/src
# scripts
project/scripts/sonar-scan.sh
# config
project/config/sonar/sonar-project.properties
# build dir, also contains Sonar output
project/build
So we are launching the scan of the src/ directory from the scripts/ dir using config/sonar/sonar-project.properties as a config.
Full details:
# The default configuration file for SonarQube
SONAR_PROP_FILE=$TOP_DIR/config/sonar/sonar-project.properties
# Sonar wrapper output dir
SONAR_OUTPUT=$BUILD_DIR/sonar
SONAR_BUILD_WRAPPER[0]=$SONAR_BUILD_WRAPPER
SONAR_BUILD_WRAPPER[1]=--out-dir
SONAR_BUILD_WRAPPER[2]=$SONAR_OUTPUT/wrapper
SONAR_SCANNER[0]=$SONAR_SCANNER
SONAR_SCANNER[1]=-Dproject.settings="$SONAR_PROP_FILE"
SONAR_SCANNER[2]=-Dsonar.projectBaseDir="$TOP_DIR"
SONAR_SCANNER[3]=-Dsonar.working.directory="$SONAR_OUTPUT/scanner"
SONAR_SCANNER[4]=-Dsonar.cfamily.build-wrapper-output="$SONAR_OUTPUT/wrapper"
Here is our sonar-project.properties:
# must be unique in a given SonarQube instance
sonar.projectKey=DspRefBase-Hexagon
sonar.projectVersion=1.0
sonar.host.url=http://virt-win-ci:9000
sonar.login=XXXXXXXXXXXXXXXXXX
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
# This property is optional if sonar.modules is set.
sonar.sources=./src
# Encoding of the source code. Default is default system encoding
sonar.sourceEncoding=UTF-8
# Analyzes only the specified files
sonar.inclusions=**/*.cpp,**/*.c,**/*.h
sonar.exclusions=**/test*/**,**/thirdparty/**/*,**/UserPresetConversion/**,**/audioengine/**/*
I doubt this is related to the bug you mentioned because I can see in the “code” tab that only changed files are present. The problem is that it is showing all the bugs present in files, not only the ones introduced in the changed lines of the short lived branch.
Hi @dextermagnific,
Short lived branches don’t hide issues depending whether a line is considered to be new or changed.
What SonarQube does is it compares the issues found in the branch (in changed files) against the issues known in its target branch, and only shows the issues that are new.
Is the old issue present in the branch targeted by this SLB?
I guess this answers my question. Can you confirm that those 74 issues are in master, in the same files?
The issues are indeed in master, but after I checked they do not have the same “age”.
In master they are showing as 2 months old, but in the short lived branch they show as 3 months old.
Something is wrong here.
The date of the issue is the last change date of the line where the issue is located.
If you use an SCM, it will correspond to the date of the last commit changing the line.
There have been many bugs fixed in SonarQube related to branches since v7.3. Is there any chance you could upgrade to 7.5?
We will try to upgrade and let you know. Thanks
Hi @dmeneses,
I think I might have a scenario that spots the issue date difference between master branch and short lived branches. I recall that short lived branch issue dates are older that the ones of master for the same issues.
Here is what we did and what I think caused the problem:
-> the short lived branch will show issues present in master but with the first date instead of the second, while master shows them with the second date.
Hope this will help you.
Hi,
Issues should be backdated on initial project analysis. We’ve made some advances in backdating since 7.3 (altho particularly in this area, to be honest). We’ve also made big improvements in branches. You really should upgrade.
Ann
Ok we will upgrade. Should we wait for 7.6 ?
Hi,
As a member of the development team, I’m always going to want to urge you to the latest/greatest. 
However, I don’t think it’s necessary in this case. The 7.6 E.T.A. is next Monday but there’s nothing related to branches in it.
Ann