JavaCheckVerifier problem when having multiple tests with multiple issues declared for the same line

Hello everyone,

I was able to declare in the tests that I expect two/more issues on the same line.
This worked very well until I have implemented another test with the same requirement (multiple issues on the same line).

Then the previous test is failing because somehow the order of the // Noncompliant tags is important and random.
All of them are being added in a collection and when an issue is detected only the last one is verified. This will fail if the order is changed and maybe the verified string is there but on a different position.

Steps to reproduce:
Clone repo:

Extend AvoidAnnotationRule class to report two issue in the same place:

  public void visitMethod(MethodTree tree) {
    List<AnnotationTree> annotations = tree.modifiers().annotations();
    for (AnnotationTree annotationTree : annotations) {
      if (annotationTree.annotationType().is(Tree.Kind.IDENTIFIER)) {
        IdentifierTree idf = (IdentifierTree) annotationTree.annotationType();

        if ( {
          context.reportIssue(this, idf, String.format("Avoid using annotation @%s", name));
          context.reportIssue(this, idf, String.format("Exception 2 @%s", name));

My test class:


import org.junit.Test;

public class AvoidAnnotationCheckTest {

  public void detected() {

    // Use an instance of the check under test to raise the issue.
    AvoidAnnotationRule check = new AvoidAnnotationRule();

    // define the forbidden annotation name = "Zuper";
	JavaCheckVerifier.verify("src/test/files/", check);

File under test:

 *This file is the sample code against we run our unit test.
 *It is placed src/test/files in order to not be part of the maven compilation.
class AvoidAnnotationCheck {

  int aField;

  public void aMethod() {

  // Noncompliant@+2 {{Exception 2 @Zuper}}
  // Noncompliant@+1 {{Avoid using annotation @Zuper}}
  public void aMethod() {

  // Noncompliant@+2 {{Avoid using annotation @Zuper}}
  // Noncompliant@+1 {{Exception 2 @Zuper}}
  public void anotherMethod() {



Now this will work but If you duplicate the test I will have errors:

org.junit.ComparisonFailure: [Line: 16 attribute mismatch for MESSAGE: {MESSAGE=Avoid using annotation @Zuper}] expected:<"[Avoid using annotation] @Zuper"> but was:<"[Exception 2] @Zuper">

Basically, the order is different and I need to switch the position of the // Noncompliant comments

I have checked the code of the sonar verify class and I found something which can be the source of the problem:

  private static void validateIssue(Multimap<Integer, Map<IssueAttribute, String>> expected, List<Integer> unexpectedLines, AnalyzerMessage issue, @Nullable RemediationFunction remediationFunction) {
    int line = issue.getLine();
    if (expected.containsKey(line)) {
      Map<IssueAttribute, String> attrs = Iterables.getLast(expected.get(line));
                                       // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ here we only take the last result
      assertEquals(line, issue.getMessage(), attrs, IssueAttribute.MESSAGE);

Is there something wrong with my tests? Is there another way to do it? I also tried with the new verifier and I have the same problem :frowning:

Thank you very much for your time, I will appreciate any help

Hey @marian31ian,

Thanks for the feedback. To make it clear and answer your question: No, there is nothing you are doing wrong in your test. The verifier simply does not support this (pretty uncommon) case, yet. I created the following ticket to not fail on it: SONARJAVA-3602

As a recommendation, I would suggest you to never report multiple issues on the same line/tree with the same rule and prefer the use of the “secondary” locations. These locations are to be added to the main issues, usually to give more context.

Finally, note that using the verifier, you can test on which lines they will be, but not the corresponding message, with the following syntax:

// Noncompliant@+1 [[secondary=16]] {{Avoid using annotation @Zuper}}
public void aMethod() { }

Hope this helps,

1 Like

Thank you for all this information. I was able to see the secondary locations in action but unfortunately I cannot test the message. In my case this is really important.

I am looking further for an update on the issue created by you inside Jira. IF, there will be a fix for this, can you tell me if it will be compatible with the LTS version we have (7.9.1 build 27448)?

The LTS embed version 6.3.2 of the Java Analyzer, and will only be updated with critical bugfixes. Unfortunately, the fix is currently planned for version 6.10 of the java analyzer, which will NOT be available on LTS and not backported. It is simply not critical enough.

However, the 6.10 version will in theory still be compatible with LTS, meaning that once released, if you are using it as a dependency in your test of custom-rules plugin, it might allow you to tests such cases, which still be compatible with 6.3.2.


It’s clear now. Thank you very much for your help :slight_smile:

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.