We are trying to achieve Quality gate to act on new code as soon as the second analysis is performed on the newly created project.
We are using SonarQube 8.9 LTS (8.9.0.43852)
Issue: When we have new project created, SonarQube is not showing the analysis on new code for next 5 hours (Time between Project Creation Time till +5 hrs)
Steps to reproduce:
check in a baseline build of any code
Build the code and get a Sonar Report
Check in changes to TFS. These should be analyzed as new code
View the timestamp on the change in Sonar (this is the bug, the timestamp doesnât match TFS)
The timestamp reflected in Sonar Matches the actual time of checkin.
New code checked in after initial build is analyzed
We tried building the project multiple times by changing the code and the scanner didnât show the changes on the new code. We see the updates on new code only after 5 Hours (UTC to CST time difference)
We had similar issue in 7.9 LTS version so we upgraded to 8.9 LTS assuming the issue could be fixed.
We also tried updating the Time zone in sonar.properties file to use UTC, America/Chicago and the default values for the below properties but didnât help.
Iâm sorry, Iâm having trouble understanding the issue. Is the problem that the timestamp in SonarQube doesnât match the one in TFS? Or is the issue that it takes 5 hours for you to see any results in the SonarQube UI?
Issue is that SonarQube is not showing Analysis on New coverage for 5 hours from the time of Initial analysis. Question was why is SonarQube not analyzing or showing analysis on new code.
Ideal scenario is when i run the 2nd build/analysis, it should show the code coverage on new code but its not happening the way it is supposed to.
I verified Server time zone, configurations and everything is the same (CST timezone).
Iâm terribly sorry, but this still isnât clear to me. So, if I try to recap:
You run an initial analysis at 1pm â quickly, SonarQube shows you the result, as expected.
You run a second analysis at 1:30pm â you see nothing in SonarQube.
At 6:30pm, all of a sudden, you see your second analysis in SonarQube. This is much later than you expect, plus the results arenât what you expect.
Is this correct? Iâm sorry if not, but I just donât understand the problem from the screenshots youâve posted.
About the 5h delay, if thatâs the main issue here, I suggest looking into how/where you run your analysis. Do you always see this 5h delay, or did it happen a single time? Could it be your CI serverâs queue? Did you check SonarQubeâs own Background Tasks queue?
I run the analysis at 1pm â I can see the project created in the SonarQube with Baseline Analysis
I run another build at 1:20 PM with new code added â SonarQube doesnât show the analysis on new code, but i can see the new code is added in the file and I can also see the overall number of lines of code count is increased.
When i run the 3rd build at 6:30 with new code added again, then iâm able to see the analysis results on the new code as shown in the screenshots above.
NOTE: I need to run the build at 6:30 inorder to see the analysis results on the newly added code. That being said, within this 5 hr period, i will not see any analysis on New Code doesnât matter how many times you run the build by adding new lines of code.
We run the analysis from TFS and both TFS database, Agent server and Sonarqube are all in the same datacenter and has same timezone. And we donât see anything from the Background Tasks. I hope you get this now.
@Wouter_Admiraal@ganncamp Can we check this issue as we are having lot of impact on the analysis and see if this has to be considered as a bug? Your help and Effort is appreciatedâŚ
We followed Alexanderâs suggestion below, it seemed to help with background tasks, try increasing postgres work_mem to 64mb (default is 4mb) assuming you are using postgres, we use aws rds for postgres so we did it via rds param group, reboot sonarqube after updating work_mem just in case.
Hello, also I would recommend considering re-analyzing the statistics for the postgres query engine, since in my case, the main problem was that the query planner was building incorrect query plans due to incorrect statistics.
We are not using Postgres and as explained above, its nothing related to memory size⌠I donât have any issues related to background task because i can see the analysis happening but the analysis on new code wasnât showing for the 1st 5 hr period (compare from base code) and unable to check with quality Gate.
What is the current new code definition of your project? You can find it under Project Settings > New code.
Could you reproduce the issue with all 3 analyses you mentioned (1PM/1:30:PM/6:30PM)? Are those analyses sharing the same version? Please provide activity page screenshots once it is reproduced.
Analysis two: Added few more lines of code and ran the analysis in less than 30 Min from the initial analysis, but the below screenshot shows 0 new lines
Analysis Three: We ran this analysis after few hours which is more that 6 hrs. Added few more lines and now it started showing the Analysis on new code.
Sonar Activity Screenshot: As you can see there is increase in the total lines between 1st and 2nd analysis which didnât showup. But in the 3rd analysis which we ran today morning (time gap is more than 6 hrs), shows the correct analysis on the new code.
Terribly sorry for the late reply, Iâm just back from paternity leave, and did not think of passing this to a colleague before I left.
Iâm re-reading all this, and Iâm still very, very confused about this â5 hoursâ thing. Analyses are a sequential thing. SonarQube doesnât care about the time between different analyses. You could send several of them only seconds apart, it will work. In fact, thatâs exactly what we do with our automated QAâs, which are run dozens, if not hundreds of times every day. The fact you have to wait 5 hours before sending a 3rd analysis, which would then only detect new code, sounds like a red herring. I doubt the timing between analyses has much to do with it, unless your Background Tasks queue is completely saturated.
Now, Iâm not saying thereâs not a bug here somewhere. But I have the feeling concentrating on this â5hâ is leading us astray, at least as far as New Code detection is concerned.
I have some extra questions:
You said you were using TFS. Are you also using TFS as your SCM (meaning, youâre not using Git or SVN)? If yes, what plugin are you using (as SonarQube doesnât provide TFS SCM support out of the box)?
Is this happening on several projects? Or only a single one? Or perhaps a specific category of projects (language, framework, etc) ?
If you submit those 3 analyses in rapid succession, do you see all 3 of them in the Background Tasks? How long before theyâre all treated by SonarQube?
Are you using DataCenter Edition? If so, do you have several nodes?
I worked on Different Sonarqube Versions before and never had this issue. All i want from you is how can i find whether its Sonarqube Bug or do i need to make any settings inorder to work. I keep saying 5 hours because any new code which is analyzed after it is treated as new code, and anything before that is never considered as new code (It doesnât matter how many times you run like 100 or 200). I have sent many screenshots with examples for the reference.
All i need is Why this is happening and how can i check and fix it.
For other questions
Yes, we use TFVC as our SCM. That being said, we are able to get the analysis on new code after 5 hr period. So i think there is no issue here.
It is happening on all the Newly added projects to Sonarqube. We are using C# with .net framework 4.7 or above, .net core 2.2 & 3.1
All background tasks are getting completed in regular manner (running about 1 min or so depending on the size of the project). As i said before, its not just 3 but as many as we run for the 1st 5 hr period.
Yes, we are using Datacenter Edition. We only have 1 node Instance
OK, and have you tried adjusting your new code period setting? Right now, I see itâs set to 400 days, which might be part of the problem (would be a bug, in that case). Iâve tried reproducing with such a high value, but no luck. Is there a reason for using 400 days as a new code period?
We tried setting it to use Previous Version but it didnât work. (Worked after 5 hr period again). So everything is revolving around this time period. We also tried setting the Java Time Zone to use CST, New York and UTC and none of them worked. Also tried commenting this setting in sonar.properties file
The ânew codeâ in a branch is always the result of comparing the last modification date of each line (usually collected by the scanner from the SCM) with a threshold which was calculated based on the âNew Code Periodâ setting. One of the dates may be wrong due to the timezone.
Could you please check the date when the New Code starts by checking the field leakPeriodDate in the response of https://my.sonarqube.com/api/components/show?component=[projectKey]&branch=[branch]
Then, for the source code file where you expected to find lines classified as ânewâ, please request: https://my.sonarqube.com/api/sources/lines?key=[fileKey]&from=[lineNumber]&to=[lineNumber]&branch=[branch]
And check the field scmDate. The file key will be shown in the URL bar when you navigate to the file in SonarQube web interface.
