When we create a new feature branch, the new code version picked up ends up showing as 7 months old or much older. our new code setting is referenced to development branch which again points to previous version. we do make weekly deployment. but whenever we create a new branch it triggers the scan and fails the build altogether. Then we end up doing multiple consecutive builds/scan which fixes it or change it manually to point to a version that has Quality gate marked as successful.
Is there something we are missing that can correct this issue?
Hello @vibinsj and welcome the SonarSource community.
First of all we’ll not be able to investigate deeply unless you upgrade to the LTS 8.9 or LATEST version of SonarQube.
Now, just a few preliminary hints: What is considered new code on any branch (including the main branch) is governed by a setting that can take up to 4 different value (4 different ways of computing the new code):
Previous version (the default)
Number of days
Reference branch
Specific analysis
Have a look at what is your setting on your project and that should probably explain why the new code period is 7 months or more.
If you can’t explain, please send your project new code setting and a (full page) screenshot of your branch issues showing some issue as old as 7 months
Thanks olivier for getting back on the query, upgrade is something thats being planned and will look to implement it sooner.
we currently have set the new code setting pointing to specific branch which points to previous version.
There are two issues here, every-time a new feature branch is created our bamboo build fails with the base scan, here it brings in issues older than 5 years which was previous marked as won’t fix.
Also on the new code it randomly picks up an snapshot that says older than current date.
So I understand that your branch where some issues are raised unexpectedly is “some-branch” (I can’t see its name in your screenshots, that’s why I always ask to provide full page screenshot to see all the information, even the info that may not seem important), uses reference branch as new code definition and the reference branch is develop.
There is a subtle distinction between definition of new code and new issues.
When you chose new code as reference branch, there is not particular moment where the new code starts. It simply diffs the branch and the reference branch. In you case I guess the branch was created 7 months ago so that’s why you new code period is said to be 7 months.
The issue you highlight is 6 months old, meaning that it did not exist (on the develop branch or on the some-branch branch. So it was necessarily marked as Won’t Fix on develop after creation of some-branch. That explains why the issue is raised on some-branch:
The manual changes on status of issues (WF, FP, Change of Severity, Type, Comments) is synchronized between branches only at branch creation time (ie first first branch analysis). In your case all WF existing 7 months ago on develop would have been propagated on some-branch. This does not happen anymore afterwards. So the WF set on develop after the branch analysis did not make it to some-branch. That’s why this issue is there.
Assuming you use git for your source control, after git checkout some-branch, if you git rebase develop or git merge develop the issues that were injected in develop after forking some-branch:
will land on some-branch as new issues (but they are not on some new code). Their status from develop (Won’t Fix or whatever) is not propagated on some-branch
Normally the beginning of the new code period in some-branch should be reset to the moment you you git merge or git rebase.
From the 2nd bulletpoint above I am therefore surprised that your new code period on some-branch is still 7 months ! I can only see 2 reasons for that:
You don’t use git as version control. SVN maybe ?
You use git as version control bt don’t merge develop changes in some-branch using git merge or git rebase. Would that be possible ?
The branches are all feature or hot-fix branches. we create a branch from development, make our changes to new feature branches and merge them back to development after PR approvals.
This branch creation is something done on daily basis by many developers and branches are named based on the feature/issue they work.
The feature branches created are relatively new and less than a week old.
we use git as a version control and we always use the latest copy of develop whenever a new branch is created.
The issue you highlight is 6 months old, meaning that it did not exist (on the develop branch or on the some-branch branch. So it was necessarily marked as Won’t Fix on develop after creation of some-branch . That explains why the issue is raised on some-branch :
The feature branch never existed 7 month ago, as its newly created last week. Let say an issue is marked as wont fix and merged to develop. will that be pushed to develop and gets propagated to any new branch created post that?
Also the baseline scan that gets done at the creation of new branch ends up with quality gate failure, shouldn’t this be passed as its the baseline and it should consider all codes from this as new.
Hello @vibinsj ,
The branches are all feature or hot-fix branches
Fair enough. That gives me more context.
Well, ok but your first screenshot of the thread mentioned a New code period of 7 months.
Can you explain ?
Generally speaking to further investigate I need more visibility on the environment of the each on each of the branches. Pick one issue that demonstrates the problem and provide:
A full page screenshot of the issue. I need to see the issue, the code around, the commit date and the last analysis date of the branch, and the message of the issue (that’s hidden by the commit tooltip)
I also need to see when the issue was marked as won’t fix on the develop branch so please add a screenshot of the issue history on that branch, click on the “issue age” to pull down the history, like below
And the history of analysis of the feature branch (to see the first analysis date of that branch, which should be after the issue was marked as won’t fix on the develop branch).
If the 1st analysis of the branch is not display on the first page, scroll down the Activity pane (left of the page) up to the bottom. I need to see the first analysis of the feature branch.
I have share few screenshots below, had to mask few items as it cannot be shared here completely.
The problem is that we end up having issues raised in files that was never touched by the developer and also issues on lines never changed by developer.
The new code setting should pick the new code added by the developer in every branch is what I believe, correct me if I am wrong.
It’s OK that you hide the project name and identity of the committer but I need to see the branch name otherwise i don’t know which branch I am looking at
I need you to stick to one particular branch and one particular issue for all this thread. The issues that you show in the last screenshots are different from the issue of your initial one.
For the moment, from you last post (with all the screenshots) here’s what I can say:
The issues shown in the first screenshot of your last post are a bit special issues. They are file based issues (ie they are not located on a particular line of code, they apply to the entire file). Indeed you can have corner case of file based issues appearing on branches, if some new code is added is added elsewhere in the file. In you case there are at least 3 new lines to cover added below in the file which could explain appearance of the file based issues on coverage.
So please select an issue (and stick to it), and issue:
that is located on a specific line of code (not a file based issue)
that is on a line of code that is not new
make a screenshot of that piece and surrounding code (please full screen) for the feature branch AND the target branch master - On the master branch pull down the issue history (should be marked as FP or WF) so that I can see when it was done.
Make a screenshot of the feature branch activity tab, and main branch activity tab as per my request on the previous post
That’s 5 screenshots in total:
Issue in code on the feature branch
Issue in code on the master branch
Issue history (pull down arrow) on the master branch
Activity tab of the feature branch (from the moment the branch was created)
Activity tab of the master branch just before the date the feature branch was created.
You can mask the projet name, the commit details (except dates) and any user identity. I don’t need that, but I need all the rest.
Without that I won’t be able to troubleshoot further. Note that there is probably no bug, just an explanation of why you may have your corner case.
1)but unfortunately I am not able to capture any recent error where the wont fix is not getting propagated. since its used by multiple developers the branch gets deleted once the build is successful. will add them once i have a issue captured
2) is there any possible explanation on why an old line of code thats created by someone else gets added in the issue (code smell/line coverage) of new code and block the build. these lines or files were never changed by the developer and its been added 1 to 5 years back. how to have it fixed?
There was only one migration that happened like 1 year back where the version upgrade of sonarqube was performed and the usage of new code setting came into existence.
Just to add, we see this issue on a daily basis where multiple feature branches are created.
Nevertheless if you want me to explain why you have this situation you have to capture the moment. When such issue happen you should have at least time to screenshot the feature branch and the target branch before the merge.
There’s always an explanation (no known bug around that) but it can be given only on a case by case basis, because there are a couple of different cases where that can happen (file based vs line based rule is one example, like the last issue you provided)
About fixing, when such an issue is raised. It’s up to you to decide to fix if deemed useful, or to ignore (First it may not break your QG, and if does you can use the Won’t fix weapon)
we didn’t face this issue before the “new code” setup came into picture, the rule would be to match existing count of issues and everything thats new and having count above this set numbers were reported.
we are having few hundred issues, that gets triggered during a later date and it’s causing lots of build failures. most of them are legacy code and are not planned to be fixed
How to check what rules or setup triggering this issue, we face both file based and line based issues as mentioned earlier.
We’ll have to solve the case of each type of each issue one at a time. As I wrote earlier, there is always a reason, but I can’t give this reason is abstraction. You have to show me the details of one type of issue and I will tell you why it is considered as new and (not always the case) if there is a way to avoid that. But first things first I need to see the issue with all the details I asked already before.
Without that I don’t think it makes sense to continue this thread.
Thanks for the screenshots.
A few reasons that can explain the new issue in that case:
The “possible god class” issue that you selected is a class/file level issue (which I already explained is a possible case where issues may be found as new) even if the file is not modified on the line it is reported. This issue being a file level one if a single line in the file is changed (anywhere), then this issue will be considered new. And you screenshot shows (understandably) only the bottom of the file. Can you check if any line has been changed between line 1 and 288 (first visible line) ?
The “possible god class” issue that you selected is a PMD issue (not a SonarQube Java analyzer one). I can’t tell why this would make any difference but this plugins is not ours and maybe it’s reporting issues in a way that (at least sometimes) does not allow us to distinguish new and old issues
I can see that you have warnings related to your analysis. This is symptomatic of a scan that works but is not set up optimally. Some of the warnings may explain why you would have issues that are not new being reported as new, namely:
– Shallow cloning in the pipeline (which may code the diffing between branches to not work)
– No SCM integration (code is not detected from an SCM) in that case SonarQube applies a best effort algorithm to determine the new code and the new issue, algorithm that is much less accurate than the SCM branch comparison algorithm.
—> Can you check what those warnings are about (click on it for more details)
[INFO] This git repository references another local repository which is not well supported. SCM information might be missing for some files. You can avoid borrow objects from another local repository by not using --reference or --shared when cloning it.
15-Jul-2021 16:58:14
[INFO] SCM Publisher 0/1 source files have been analyzed (done)
15-Jul-2021 16:58:14
[WARNING] Missing blame information for the following files:
15-Jul-2021 16:58:14
[WARNING] * pom.xml
15-Jul-2021 16:58:14
[WARNING] This may lead to missing/broken features in SonarQube
I gave you 3 possible reasons for your problem and you reply only focusing on one. Please make sure you take into account all my feedback. Bulletpoints 1 and 2 are also important and I am certain they have an impact
