Quality Gate failing despite no code change

We recently started using SonarCloud for Bitbucket cloud repository with quality gates on Jenkins CI builds, and set up our New Code definition to be anything after April 22nd. Last night our Java quality gate failed despite the class in question not having been touched since the start of last year.

The rule in question is javasecurity:S5146, which looks to have been available since March 26, 2019.

From checking the activity on our project, I can’t see anything that would indicate this rule changing status.

Has anyone experienced anything like this?

HI @Ben_Green

can you share screenshots about your project, Quality Gate settings etc?

Carine

Hi @Carine_Bayon. Thanks for the quick reply. Yep, I can share some screenshots. What specifically would you like to see?

Thanks,
Ben

The homepage of your project, where you see that QG has failed, and the settings of your QG criteria could be a good start.
Thanks

Attached is a picture of the home screen with specific names and metrics hidden. You can see there that the quality gate has failed.

The second picture is the rules of our quality gate. Thanks in advance @Carine_Bayon.

Can you also send a screenshot of the issue there? and activity tab?
You should have changed something there, that raised a New Blocker issue, that made the QG failed today.

Carine

Here’s the activity tab. As you can see, the quality gate failed this morning.


Here’s the summary of the new blocker:

However, this is the line of code in question. As you can see, it hasn’t changed since 2020.
code-summary

Hi @Ben_Green,

Welcome to the community! This type of thing can happen when a file in the flow is modified. Let me give you a simple example.

Let’s say this code is all dated 1 January 2021. It’s all old code:

1  public void doTheThing(String foo) {
2    if (foo != null) { 
3      System.out.println(foo.length());
4    }
5  }

As you see, there are no null pointer dereference problems here.

And then there’s an edit:

1  public void doTheThing(String foo) {
2      System.out.println(foo.length());
3  }

The lines that are left are still all dated 1 Jan 2021. They’re all still old code. But there is now a new null pointer dereference issue, which will be raised as a new issue on old code.

With your issue I suspect something similar - something in the flow of the issue changed to create a new issue on that untouched code. If it’s an addition you’ll be able to tell when you look at the secondary locations associated with the issue. If it’s a deletion, it will be tough to track down.

 
HTH,
Ann

I had quality gates fail this morning too on the JS engine. Javascript:S2819 for me on a file that hasn’t been modified since 1st December 2020.

Had a Bitbucket pull request that was showing as zero issues, then I merged it to master and the Quality Gate failed.

Even I had the build gates failing with the error “ERROR: The ‘files’ list in config file ‘tsconfig.json’ is empty.” And when looked in the sonar cloud I see the master branch had major/minor code smell on the file that were never changed since the year 2020.

Hi @ganncamp,

Thanks for your reply. I work with Ben.

I can confirm that the file that is now reporting the new violation has not changed since Feb last year.

regards

Ramon

To back up what @padillr has said, looking in the git log for that file there has not been any commit containing an addition or deletion to that file since June 2020.

Hi all,

I had hoped to make clear with my example that changes in other files can cause legitimately new issues to be raised on old code.

@bharatpidapa your issue seems to be different. You need to start a new thread.

For everyone else, can you take a serious look, please at what other files might have changed to cause your new issues?

 
Ann

Just to confirm we are getting this issue too, we are getting security hot spots and code smells thrown on old code, On files we definitely haven’t touched at all in the new code period.

We are also getting the same tsconfig error as @bharatpidapa but that is from a third party task called ‘SonarCloud build breaker’ in Azure Devops, but does lead me to believe that something fundamental has changed behind the scenes with the sonar scanners.

Edit: Also worth nothing that these code smells were not flagged on the pull request branches, only on the master branch once the merge has happened.

Yes, something has changed in the behind. We started to see the Azure CI build pipeline fail due to missing tsconfig file and when looked into the Sonar cloud we see that the master branch has new code smell on the files that were not changed at least since the year 2020.

Hi @ganncamp. Sorry for not following. How would a change to another file effect the analysis of an existing file? Maybe I don’t understand how Sonar works, but I’d assumed that each file was analysed in isolation.

Hi @Ben_Green

In my code example above, I demonstrated that a change to another line to cause the creation of a new issue on old code.

Let’s take it one step further:

//File A
1  public void make the call(String foo) {
2    if (foo != null) {
3      doTheThing()
4    }
5  }

//File B
1  public void doTheThing(String foo) {
2      System.out.println(foo.length());  // no issue here
3  }

Now compare that to this version:

//File A
1  public void make the call(String foo) {  // We deleted lines in a different file!
2      doTheThing()
3  }

//File B
1  public void doTheThing(String foo) {
2      System.out.println(foo.length());  // New null-pointer dereference issue on untouched code
3  }

Hopefully this makes clear how new issues can happen in old code.

 
Ann

Thanks for that explanation @ganncamp, so the analysis is essentially running the code and “new code” is treated as any code that has been affected by the lines you have changed, not just the actual lines that have been changed.

Has this always been the case? Something must have changed within the last few days with a release of one of your plugins to cause a change. There are multiple threads currently open in the new section with people all complaining about exactly the same thing, that isn’t just coincidence!

Also, why are these issues not flagged on the PR branches, I don’t want our developers being told they have a passing quality gate on the pull request, to then get the code into master to be told master is now failing, this is now blocking our team until someone goes back and fixes code that could have been sorted in a pull request.

1 Like

Thanks @ganncamp. I’m still not entirely convinced that’s the case for us. The class in question is a servlet, and is only called in on class - The unit tests for it. Those tests again haven’t been touched since February 2020.

To echo @pconnor, if there were an error I’m surprised we didn’t see anything in a pull request before seeing the quality gate fail in master.

Hi,

@pconnor, that’s not quite right. New code is code added or edited in the New Code Period. New issues are issues raised in the New Code period. And as I’ve hopefully demonstrated, you can have one without the other.

Fair question. PR analysis only raises issues on new code and will miss new issues on old code.

@Ben_Green what changes have been made to your system recently? Did you recently upgrade? If so, it’s entirely possible that javasecurity:S5146 got smarter. It’s quite common for rule implementations to improve from one version to the next.

 
Ann