I’m using SonarQube 7.0.0.36138 and I’ve configured a leak period using a custom date (2018-09-01) combined with a quality gate that focuses on new issues. However, when the analysis runs, I’m seeing issues bugs and vulnerabilities being reported as “new” that, while are valid issues, were not created since the leak period custom date. This leads to failing the quality gate.
I’m a bit unsure of this status of these issues and whether they’re the same as what I’m experiencing, but any advice / assistance around getting this configuration working would be greatly received.
To help, we’ll need a few more details. First, what version of SonarQube are you using? Also, take a couple of new-old issues, and list out, please the last-modified date for the line each one is raised on, as well as the date the issue is given.
Also, you should know that when you set a date as a new code period, the first analysis inside (so after midnight) that date is used as the baseline for comparison. Everything after that is “new”. That’s really tangential to your question, but worth mentioning IMO.
As mentioned in my report, I’m using SonarQube 7.0.0.36138.
The particular issue that brought this to my attention states “September 4, 2018 10:14am Created”.
However, I can see in source control that there have been no changesets for this file since May (when the branch was created).
In terms of a baseline build, my expectations were as you described. The first build in the leak period would identify all issues as existing with none being new. My first build in for this project was on September 3, 2018 (see below, taken from the Activity tab of the project).
September 4, 2018 10:14 am Quality Gate Red (was Green)
September 3, 2018 4:13 pm (no mention of quality gate analysis)
August 31, 2018 1:40 pm Quality Gate Green (was Red)
So, what this reveals is that:
The code file in question didn’t cause the quality gate to fail on August 31 (before the leak period date).
The September 3rd analysis doesn’t have a quality gate result, which I assume is because it’s the baseline.
The September 4th analysis has picked up an issue that existed in the previous builds analysed as the code file has been unchanged for months, but has indicated it is new.
Additionally, I’ve not enabled/disabled any rules over this period of the above analysis.
With regard to reviewing other issues, it’s a pretty consistent problem. I have many files that should have been analysed on both the 31st August (before the leak period) and 3rd September (within the leak period) and many analysed builds before that, that haven’t changed since the branch was created, which are now reporting new issues in the 4th September analysis.
If you need any more information, please let me know.
Sorry I overlooked the SQ version in your initial message.
What rule(s) are these issues being raised against?
Personally, I had a slew of “unused import” issues raised against rather old code of mine just yesterday; another developer had removed the uses of those imported classes. So I’m wondering if this could be a similar case. Alternately, if your exclusions/inclusions have changed in the meantime, that could bring old code under analysis and produce “new” issues.
As you can see, the screenshot shows the issue is 2 days ago (after the leak period). TFS shows the file check-in as 30th March 2018.
I’ve also got the same occurring on a useless assignment to local variable (javascript:S1854) for 3rd party JavaScript library and another for an invalid string format (csharpsquid:S2275).
It does sound like in your scenario, the code file had been edited (when the developer removed uses of an import from the code file? In my case, they haven’t. There are no change sets or check ins for the code file since the leak period so they should have been captured in the baseline? In addition, the none of the rules have been edited since the start of the leak period date, so I’d expect everything to be measures against the baseline build.
The project has been analysed many times, we use CI to perform a nightly build of this project and check-ins are daily (except for weekends):
September 4, 2018 10:14 am Quality Gate Red (was Green)
September 3, 2018 4:13 pm (no mention of quality gate analysis)
— September 1st: Start of Leak period
August 31, 2018 1:40 pm Quality Gate Green (was Red)
— Many builds, all analysed by SonarQube
With regard to inclusions/exclusions, the C# Quality Profile was last updated 2 months ago (25th June, well before the start of the leak period) and it’s the same as the Sonar way rules with one exclusion. The other profiles (javascript etc.) are all using the default [built in] rules.
The plot, however, now thickens! We’ve just done a wave of builds for multiple projects and I’ve noticed all of the quality gates have now gone green for the project we’ve been discussing (yet these were nuget updates only, no changes to any of the code files mentioned). Something I have noticed, when I now review the project in SonarQube and click on the Code tab, I’m seeing code listed for a different project! On face value, it looks like I may have 2 projects building with the same project key. I’ll go review it now and see if that’s just throwing everything out of place.
So, I’ve checked the build configurations for both projects, specifically the prepare analysis steps:
Project A is the project we’ve been discussing, this is it’s project key: admin.short:Short
Project B is contains the code now displayed in project A in the SonarQube site, it’s project key is: services.short:Short
These values are consistent with what I see in SonarQube in Administration -> Projects -> Management in the “Key” column. They all appear to be correct.
Now, as mentioned, we did kick off a wave of builds earlier to our development environment (we have multiple pipelines and build agents). This occurred simultaneously. Could there be a problem with SonarQube receiving multiple builds for analysis simultaneously? Possibly mixing up some state data from the two builds and thus confusing the analysis?
I’ll give the builds another review to make sure they’re all correct, but if there’s anything else you think I could/should check, please let me know.
When the server receives analysis reports, it queues them, then works through the queue. I don’t see how data from two different analyses could be mixed together.
Is there anything you can think of, other than using the same project keys in two different projects, that could cause this? Maybe the TFS prepare task? The analysis assemblies and where it stores files in the build agent?
I’ve just performed another build (with 0 code changes) of Project A. SonarQube is now correctly identifying the original issues, and code, that had “disappeared” when it became confused with Project B.
Here’s a screenshot of the activity for the project:
This shows that the 11:53 analysis (which appeared to contain code for the wrong project) suddenly became green.
This screenshot shows the activity “duplications” chart. What’s interesting is that, quite often, hundreds of thousands of lines of code disappear and reappear from the project:
It’s also interesting that it seems to jump up and down between the same two points, suggesting the confusion between Project A and B has happened before, and very intermittently, particularly since we build pretty much daily.
(Apologies if I’m bombarding the thread, as I reveal information that I think may be useful to help us get to the bottom of this, I’m posting it).
I’m afraid this is going to be on your build infrastructure side. Do you have logs of previous analyses that would allow you to see exactly what values were passed to analysis? If env vars are used, it could be that it’s getting scrambled there.
If you’re absolutely sure it’s not, then it’s probably time to open a new thread, since this one has suddenly started to drift a bit far from “configuring a leak period with custom date…”.
Fundamentally, my problem is still getting the leak period to work. It’s only through these discussions and troubleshooting it’s beginning to look like the root cause of why the leak period isn’t working might be because the analysis is becoming mixed up. If you need me to open another thread to track it further, I’m happy to do that, but this is all context to my fundamental problem.
I’m in the process of reviewing my TFS logs and I can’t find anything suggesting it’s to do with configuration of the build tasks (prepare / run / publish analysis).
The only build of project B that I can have that occurred just before the 11:53am analysis occurred 16 minutes before, here are some snippets from the log file:
2018-09-06T10:27:44.7153723Z Task : Prepare Analysis Configuration
2018-09-06T10:27:44.7154190Z Description : Prepare SonarQube analysis configuration
2018-09-06T10:27:44.7154590Z Version : 4.3.2
2018-09-06T10:27:44.7154952Z Author : sonarsource
2018-09-06T10:27:44.7155437Z Help : More Information
2018-09-06T10:27:44.7155954Z ==============================================================================
2018-09-06T10:27:48.9853231Z [command]C:\agent2_work_tasks\SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157\4.3.2\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe begin /k:services.short:Short
2018-09-06T10:27:49.9849041Z SonarScanner for MSBuild 4.3.1
It’s correctly used the project key services.short:Short
2018-09-06T10:44:12.2458277Z INFO: ANALYSIS SUCCESSFUL, you can browse http://myserver/dashboard/index/services.short:Short
2018-09-06T10:44:12.2459555Z INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
2018-09-06T10:44:12.2460762Z INFO: More about the report processing at http://myserver/api/ce/task?id=AWWue3g7TMTUfYkRzFpI
2018-09-06T10:44:46.5465649Z INFO: Task total time: 2:47.499 s
2018-09-06T10:44:46.6569509Z INFO: ------------------------------------------------------------------------
2018-09-06T10:44:46.6570382Z INFO: EXECUTION SUCCESS
2018-09-06T10:44:46.6571156Z INFO: ------------------------------------------------------------------------
2018-09-06T10:44:46.6593308Z INFO: Total time: 2:49.863s
2018-09-06T10:44:46.7939030Z INFO: Final Memory: 20M/520M
2018-09-06T10:44:46.7939726Z INFO: ------------------------------------------------------------------------
2018-09-06T10:44:46.9398541Z The SonarQube Scanner has finished
2018-09-06T10:44:46.9471846Z 10:44:46.946 Creating a summary markdown file…
2018-09-06T10:44:46.9528249Z 10:44:46.952 Analysis results: http://myserver/dashboard/index/services.short:Short
Well, I understand the “leak period problem” now (BTW, 7.4 updates that to “New Code period” since it requires so much less explanation):
p1 code change 30 May
analyse p1 -> issue i1 opened as new issue with current date
…
analyze p2 as p1 -> issue i1 closed (no longer found in code)
goto 2
This explains why your old issues are showing up as new without code changes. BTW, 7.4 (E.T.A. end Sept.) fixes this open/close/open-new-issue problem for issues closed <30 days ago by reopening the old issues.
With regard to project B code and analysis being publish to project A, would you like me to start the new thread? I’ve provided information above detailing that, as far as I can tell, project B was correctly published to SonarQube with project B’s project key, but appeared in project A, leading to this problem?
I can confirm that I’ve subsequently built both projects, in isolation, via TFS and the analysis has correctly published to the correct projects in SonarQube. I’ve not needed to edit the build definitions in any way to achieve this.
**Edit: also, if there’s a query I can run or some data I can obtain from the SonarQube database that can help identify what SonarQube did when performing the specific project analysis, please let me know.
If you really want to pursue the topic, then that would be best, but I can pretty much guarantee you this isn’t happening on our side. This is in your build infrastructure.
Fair enough, if you’re sure it’s our build infrastructure, there’s little point starting a new thread. If you could give me a clue where to look I’d greatly appreciate it as the TFS logs are telling me it looks correct.
Update: I’m reviewing all 22 of our SonarQube projects and I can now see it’s happening to others too.
