New Code measure doesn't include some new vulnerabilities reported by PVS

  • Versions used
    • SonarQube Enterprise Edition Version 8.9.3 (build 48735)
    • SonarScanner 4.7.0.2747
    • Plugin: PVS-Studio plugin version 7.12
  • Issue: a new vulnerability detected by the PVS-Studio plugin is not taken into account by SonarQube, in the context of New Code.
  • Steps to reproduce:
    Given a C++ file that looks like (the line numbers & comments are added for explanatory purpose, they are not versioned in Git):
line 10: if (foo == nullptr) { // This line was modified in the last Git commit. This line is effectively New Code.
    ...
line 20:   *foo = bar;      // This line was last modified 3 years ago
}

When using the PVS-Studio analyzer, it correctly detects that there’s a new vulnerability at line 20, as a result of line 10 being changed (it was originally written as if (foo != nullptr)).
After publishing the results using the sonar-scanner, the new vulnerability is not shown in SonarQube. I’ve tried using all the different New Code settings: Previous Version, Number of days, Reference branch.
Here is the command line that I used:

sonar-scanner -Dproject.settings=sonar-project.properties -Dsonar.projectVersion=66 -Dsonar.projectBaseDir="J:\jenkins\workspace\Genesis_PR-602" -Dsonar.pvs-studio.reportPath="J:\jenkins\workspace\Genesis_PR-602/pvs-results-merged-66.plog" -Dsonar.pvs-studio.sourceTreeRoot="J:\jenkins\workspace\Genesis_PR-602" 

And I can see that the PVS Studio plugin is loaded:


[2022-03-03T18:33:56.830Z] INFO: Reading PVS-Studio Report: J:\jenkins\workspace\Genesis_PR-602\pvs-results-merged-66.plog
[2022-03-03T18:33:56.830Z] INFO: Parsed messages: 218, Skipped FalseAlarms: 0, Unparsed messages: 1
[2022-03-03T18:33:56.830Z] WARN: Number of removed duplicated messages: 27
[2022-03-03T18:33:57.343Z] INFO: Sensor PVS-Studio Issues Loader Sensor [pvsstudio] (done) | time=897ms

For example, when using the Reference branch, and using the oldest analysis of the develop branch, the vulnerability is present in the Overall code section, but not in the New Code one.

Am I correct that the current behaviour is explained by the fact that the line where the vulnerability is reported was last modified 3 years ago, hence New Code ignores it? If this is the case, then this is a bug, in my opinion. A code change could introduce vulnerabilities in old code, SonarQube should report this.

The same issue is present for PR analysis.

Hi,

Welcome to the community!

At all? Or do you mean it’s not reported as a “new” issue? Is this in the context of PR analysis? If so, that would explain it. We’ve made a conscious decision to filter issues reported in PRs to only issues raised on new code.

If the context isn’t PR analysis, is this the first analysis after an upgrade or plugin installation? In those cases, issues are backdated.

BTW, in case you’re using Community Edition, you may be interested to know that Developer Edition($) starts at $150/year and includes C++ analysis, which catches that issue.

 
Ann

Hi,

At all? Or do you mean it’s not reported as a “new” issue? Is this in the context of PR analysis? If so, that would explain it. We’ve made a conscious decision to filter issues reported in PRs to only issues raised on new code .

The new vulnerability is not shown at all in SonarQube in the context of a PR analysis.

If the context isn’t PR analysis, is this the first analysis after an upgrade or plugin installation? In those cases, issues are backdated.

The issue is also not shown for a branch analysis in the New Code section (despite the fact that the issue is present in Overall Code, and is not present in the Reference branch of the Previous version). This is explained indeed by the backdating feature, I was not aware of it.

We’ve made a conscious decision to filter issues reported in PRs to only issues raised on new code .

Is there some configuration to relax this behaviour? E.g. to include new issues raised in modified files instead of strictly new code. As it is, this is currently a severe limitation for us, which questions the reliability of PR’s Quality Gates.

BTW, in case you’re using Community Edition, you may be interested to know that Developer Edition($) starts at $150/year and includes C++ analysis, which catches that issue.

Could you please explain why the Developer Edition would catch the issue? Even if the issue is not “raised in new code”? Also, backdating doesn’t apply in this case?

Edit: we are using the Enterprise Edition in our company.

Thanks,
Cristi N

Hi Cristi,

We’ve made an effort to only report issues on new code in PRs, so in the context of a PR, it’s not at all surprising to me that the issue wasn’t raised. That said, I think that we do show issues raised on old code if a secondary location is new code. (So for instance if the issue on line 20 had listed line 10 as contributing.) Although I’m not confident enough to swear on that.

If it’s backdated, then it wouldn’t.

There is not. To be honest, we’ve struggled a bit with where to put the line on this and we’re continually assessing. I’ll pass your feedback on internally to further fuel the fires.

I didn’t mean to imply that DE would raise the issue in a PR if PVS doesn’t (altho if it marks line 10 as a 2ndary location, it might (see quibbles above)). It’s still subject to the same structure. I’m just saying that it raises the issue too.

Cool, so then you already have access to the native rules. If you haven’t given them a go yet, you might do that.

 
Ann

Thanks Ann, the “new code” feature is clear to me now. I believe some improvements would be welcome in this area, e.g. allowing the user to customise the feature.

We can close this ticket,

Thanks,
Cristi