SonarQube Enterprise Edition Version 10.3 (build 82913)
Deployment: Installation on virtual machine
On our production SonarQube server we run both full scans and pull request scans. We also have 5 custom rules defined. One of these rules only reports results in a full scan and is not reporting issues for pull request scans (where the changes seen by the pull request have the issue present). I have tested two other custom rules and they work as expected in full and pull request scans.
Full and pull request scan are run using the sonar (java) CLI application using a sonar-project.properties file for configuration. The same quality profile is used for both jobs which selects the same rules for analysis.
I de-compiled the custom rules and I see the failing rule implements the org.sonar.api.batch.sensor.Sensor interface, and the working rules implement the org.sonar.plugins.java.api.IssuableSubscriptionVisitor interface.
Is it possible that ‘Sensor’ rules are incompatible with pull request analysis? Is there something particular that must be configured to make a rule compile with pull request analysis? These rules were made years ago.
If ‘Sensor’ rules are not an issue, how would I debug this problem to identify why a specific custom rule will not report errors correctly for pull request analysis?
For this 1 custom rule – are the issues you expect to be raised on changed lines in the pull request, or could it be that you’ve defined a file-level issue, which isn’t supported in a PR context.
I do expect the issue to be reported based on differences. In my test case I created a new file with several issues (like unused import) and the rule did not produce any issues with the new file.
I am not sure about what a “file-level issue” means. This could be the case. Looking at the custom rule java code I see the following for reporting the error if I read the custom rule scanning code correctly. Should the error be reported differently to qualify for pull-request scan?
To get more specific the rule is checking for the presence of a specific copyright header using a custom rule. I noticed that there is now a similar rule in the default java sonar rules (“java:S1451”
named “Track lack of copyright and license headers”). I also tried configuring this rule to see how it would work by setting a regular expression the the rule to match our copyright. I have not yet perfected the regex to match our copyright format, but I see that this rule behaves in the same way, I see failures when run in a full scan but I see no errors reported on new files added without a copyright header in a pull-request scan.
A file-level issue implies that the issue location isn’t associated with a specific line but with a file overall. S1451 is an example of a rule that behaves exactly this way.
That being said, the code snippet you shared:
Would imply an issue with a specific line location.
When these issues get raised, what does compare.getLine() typically resolve to? I’m wondering if it could be some funky value, like 0.
All of this said, I think it’s going to be hard to help debug the issue without a solid reproducer (code that you expect to raise an issue, and the custom rule that isn’t raising an issue in a PR context).
I checked the logic and the line number returned is -1 so the problem is as you suspect. I made a change to set the line to ‘1’ and now the pull-request analysis is reporting the error.
This change however broke the unit test code supplied with the rule for validation and I am now stuck trying to figure out how to update the unit test based on the sonar plugin test framework.
The tests used the JavaCheckVerifier provided in sonar-java-plugin version 5.10.1.16922 (this plugin was written some time ago and not migrated to the newer API’s). With this framework the test failed complaining about the line number setting I changed and it is not clear to me now to fix this.
I attempted to upgrade sonar-java-plugin to version 8.1.0.36477 which required replacing JavaCheckVerifier with CheckVerifier. I did this and got the test running again but now the tests fail with messages like the following.
ERROR: 'assertOneOrMoreIssues()' is called but there's no 'Noncompliant' comments. In file (BadCopyright.java:1)
Looking at the sonar plugin documentation it seems the test framework is tuned around test driven development and expects a “// Noncompliant” comment marking in the test input file to mark where the errors should be found. However in my rule the errors are IN THE COMMENTS, more specifically in the copyright comment header.
Is there a way to write a unit test using the CheckVerifier framework and use some other mechanism for indication of where the error is? Note in at least one test the error is the absence of a copyright header so how would that be handled in the CheckVerifier style of rule validation?
I suspect I will need to create a non-sonar-framework test to validate this functionality.
For now I just created my own test harness and abandoned the JavaCheckVerifier to get my unit tests working. I also postponed update of the sonar API to complete and deliver the fixed custom sonar rule for now.