Quality gate doesnt appear to be working

SonarQube Server / Community Build
1

We have sonarcloud installed and a quality gate configured however it doesnt seem to be catching issues. I made a contrived example of duplicate code yet the quality gate still shows passed. Here is a small bit of the code:

image
image1285Ɨ912 50.9 KB

And here is the quality gate showing as fully passed:

image
image1433Ɨ772 52 KB

Is this a bug or am I misunderstanding what kind of duplicate code sonarcloud finds?

2

Hi,

Could you share whatā€™s in your Quality Gate? Also, what language are we looking at? The algorithm varies slightly by language.

 
Ann

3

Thanks Ann. This is C#. Here is the quality gate:

image
image943Ɨ658 32.6 KB

4

Hi,

Thanks for that. Could you also share the analysis log?

The analysis / scanner log is whatā€™s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

 
Thx,
Ann

6

Hi,

Unfortunately, I canā€™t get to pastebin.com (corporate security). Can you simply upload the log here, please?

 
Thx,
Ann

8

Hi,

Thanks for the log. I can see duplication detection taking place:

But not much more than that. TBH, Iā€™m not sure of the significance of ā€œ119 files had no CPD blocksā€.

Can you verify that the lines changed in your PR show up as ā€˜newā€™ in your PR analysis?

Could you also add /d:sonar.verbose=true to your begin command and provide that log?

 
Thx,
Ann

9

Thanks, Iā€™ll send that debug log to you directly as itā€™s too large to redact.
Edit- I dont actually see a way to start a private message in this forum, whatā€™s another way I can get this log to you?

Yes, they show up as ā€˜newā€™ in the analysis but no duplicates:

image
image1660Ɨ784 68.8 KB

10

Hi,

Taking a fresh look at this, and consulting the docs I suspect your blocks may be too small to show up as duplications. Quoting:

For a block of code to be considered as duplicated:

ā€¢ Non-Java projects: There should be at least 100 successive and duplicated tokens. Those tokens should be spread at least on ā€¦ 10 lines of code for other languages

 
Ann

11

Thanks Ann, I changed the test code to be about ~500 duplicates of ā€œConsole.WriteLine(LINE_2);ā€ and it still passes showing no duplicates. I have the extended debug logs but not sure how to get it to you as I wont be able to post it publicly.

image
image1671Ɨ889 68.4 KB

12

Hi,

Thanks for doing the experiment. Iā€™m going to call for reinforcements.

 
Ann

1 Like
16

@az54263

Could you please give me the path of the files where you would expect the duplication to happen (here or the existing DM)?

Also, would be great if you could try to test this with a toy project as well with maybe 2-3 identical classes in different files to see if the issue reproduces with your current analysis setup.

Thanks!

17

Thanks Mary, Iā€™ll DM you

18

From the logs the file seems to be included in duplication detection. To go further Iā€™ll need to be able to reproduce this with code locally to understand what is going on. As requested in DM it would be great if you could provide me at least the cs files stripped of any sensitive info so I can try to create a project and replicate based on them.

My hunch is that these output prints in your print screens are not detected as duplication.
What you could do to also test this assumption is to copy-paste 2-3 times in your src directory a cs file containing the same class, same methods but different name and open a PR with this change only. This should trigger the duplication QG failure as the content will be identical and it wonā€™t be console outputs.

Thanks!

19

Thanks for your help Mary,

I ran the experiment and created a new branch with only changes being 4 instances of the same method with only the name changed but this is the same result- quality gate passed and shows 0% duplications. This file is in a class library running .NET 8. Iā€™ll redact my test file and provide it to you by DM.

I want to clarify- Iā€™m not certain this is only an issue with duplicate detection. We havent had an failures of the quality gates at all, including some that I believe should have triggered ā€œsecurity hotspotsā€. The quality gate may not be processing at all. I am showing the issues with duplication here because it seemed like an obvious case of the quality gate not working and a good avenue for debugging.

That said Iā€™m not actually sure how the security checks work so maybe that piece is behaving normally. If you have a piece of example code which should definitely trigger a security issue I can insert it and see what happens.

1 Like
20

Hello @nagarjuna183!

Ok so I might have managed to reproduce the issue.
Duplication needs a fair chunk of duplicated lines to kick in. For me it kicked in when I introduced the following class twice with different names in a single PR (new code):

public class CalcExt
{
    public double Add(double a, double b) => a + b;

    public double Subtract(double a, double b) => a - b;

    public double Multiply(double a, double b) => a * b;

    public double Divide(double a, double b)
    {
        if (b == 0)
            throw new DivideByZeroException("Cannot divide by zero.");
        return a / b;
    }

    public double Modulo(double a, double b)
    {
        if (b == 0)
            throw new DivideByZeroException("Cannot modulo by zero.");
        return a % b;
    }

    public double Power(double a, double b) => Math.Pow(a, b);

    public double Sqrt(double a)
    {
        if (a < 0)
            throw new ArgumentException("Cannot take square root of a negative number.");
        return Math.Sqrt(a);
    }

    public double Abs(double a) => Math.Abs(a);

    public double Max(double a, double b) => Math.Max(a, b);
}

Then for sure you can try to introduce on purpose a security hotspot (Pick any from the list and just replicate the non-compliant example) to narrow down the issue even more.

Two more things:
Can you please make sure that the Quality Gate assigned to the project you analyze is the expected one?

If yes, please make sure your expected conditions are clearly set for overall code (main branch usually) or new code (PR analysis).

Thanks!

26

Thanks Mary, I was also able to reproduce using this approach. Iā€™m a bit bewildered the other earlier examples with hundreds of duplicates didnā€™t trigger it, but I suppose thatā€™s fine as long as everything is working to spec.

1 Like