Update SonarJava to 5.4 - SonarLint detects no custom issues

eclipse
java
custom_rules
sonarlint
sonarqube

(Alex Schumann) #1

Hi,

I’m developing a custom rules plugin. The rules are working and both SonarQube and SonarLint are showing me the issues.

SonarQube 6.7.1 (but also replicable with 7.1)
SonarJava 5.1, 5.2, 5.3
SonarLint 3.4, 3.5, 3.6
Eclipse 4.7.1a, 4.7.3a

However as soon as I update SonarQubes SonarJava to 5.4(.0.14284) SonarLint stops showing me issues of my rules.
SonarQube still finds everything (by running maven “sonar:sonar” - I’m really not used to maven).
Lint still shows issues of SonarWay-rules but not any of my rules.

I got only 9 Rules active for testing and in the SonarLint console it even says:

"Quality profiles:

  • java: PWQo[…]iVb (9 rules)"

As seen above I pretty much tried every possible combination but could not get it to work with SonarJava 5.4. Of course I also tried a lot in the pom.xml of my plugin. I think I’m missing something obvious here.

What could be wrong?

Regards
Alex


(Michael Gumowski) #2

Hello Alex,

Their have been many changes in SonarJava v.5.4, but I can not find any change which would have impacted support of custom rules. So I’m a bit surprised of such side effects.

  • Did you tried updating dependencies of your custom plugin to rely on SonarJava 5.4.0.14284?
  • Would it be possible to share your custom plugin (privately maybe?) It would allow us to investigate the issue on our side.

Cheers,
Michael


(Alex Schumann) #3

Hello Michael,

Thanky you for your prompt reply.
Yes, I updated the dependencies of SonarJava of my project. I think I updated every version I could find in the pom.xml.
Could I just give you the pom.xml? As mentioned in my post I’m not well experienced in maven so I think/hope the problem is there.

Alex


(Michael Gumowski) #5

You can share the (anonymized if needed?) POM if you want, but I’m not sure we will be able to find something from it on the long run.

From my point of view, if there is an issue, it’s going to be on SonarJava side and the interaction with the custom plugin itself. Maybe rule registration in rules definition. So having the plugin itself would allow us to look at what is happening when triggering the custom rules (if even present in SonarJava 5.4). But let’s start first with the pom!

Note that I will be OFF for the a few days starting tomorrow, so I won’t be able to answer furthermore on the topic. I’ll ask around if someone else from the dev team can have a look in the meantime.


(Alex Schumann) #6

I reduced the project so I could give it to you (the problem of course/unfortunately still persists).
But how should I send it? Is there a way to send private messages or even files via the new website?


(Nicolas Bontoux) #7

Have you tried the upload button ?

Or you could even drag-drop the file in the text input field.


(Alex Schumann) #8

Sorry, but is there a non-public way to do that? :see_no_evil:


(Nicolas Bontoux) #9

Nope, this forum is an open/community one. And as you can imagine there are shared benefits to work based on generic/minimal reproducers that can be viewed/used by all, especially in the context of a community forum like this one.


(Alex Schumann) #10

hmm, that’s true, of course.
I tried uploading it but i’m only allowed to upload images.

4%20-%20SonarLint%20detects%20no%20custom%20issues%20-%20Plugin%20Developmen

Maybe it’s that simple and you find it in the pom:

pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>VectorSonarPlugins</groupId>
  <artifactId>vi.sonar.rules.plugin</artifactId>
  <packaging>sonar-plugin</packaging>

  <name>VectorRuleset</name>
  <description>Java Rules for SonarQube by Vector Informatik GmbH</description>
  <inceptionYear>2018</inceptionYear>

  <properties>
    <sslr.version>1.22</sslr.version>
    <gson.version>2.8.5</gson.version>
    <sonar.version>6.7</sonar.version>
    <sonarjava.version>5.4.0.14284</sonarjava.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.sonarsource.sonarqube</groupId>
      <artifactId>sonar-plugin-api</artifactId>
      <version>${sonar.version}</version>
      <scope>provided</scope>
    </dependency>

    <dependency>
      <groupId>org.sonarsource.java</groupId>
      <artifactId>sonar-java-plugin</artifactId>
      <type>sonar-plugin</type>
      <version>${sonarjava.version}</version>
      <scope>provided</scope>
    </dependency>

    <dependency>
      <groupId>org.sonarsource.java</groupId>
      <artifactId>java-frontend</artifactId>
      <version>${sonarjava.version}</version>
    </dependency>

    <dependency>
      <groupId>org.sonarsource.sslr-squid-bridge</groupId>
      <artifactId>sslr-squid-bridge</artifactId>
      <version>2.7.0.377</version>
      <exclusions>
        <exclusion>
          <groupId>org.codehaus.sonar.sslr</groupId>
          <artifactId>sslr-core</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.codehaus.sonar</groupId>
          <artifactId>sonar-plugin-api</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.codehaus.sonar.sslr</groupId>
          <artifactId>sslr-xpath</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.slf4j</groupId>
          <artifactId>jcl-over-slf4j</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.slf4j</groupId>
          <artifactId>slf4j-api</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

    <dependency>
      <groupId>org.sonarsource.java</groupId>
      <artifactId>java-checks-testkit</artifactId>
      <version>${sonarjava.version}</version>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>com.google.code.gson</groupId>
      <artifactId>gson</artifactId>
      <version>${gson.version}</version>
    </dependency>


    <dependency>
      <groupId>org.sonarsource.sslr</groupId>
      <artifactId>sslr-testing-harness</artifactId>
      <version>${sslr.version}</version>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.7.25</version>
    </dependency>

    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.assertj</groupId>
      <artifactId>assertj-core</artifactId>
      <version>3.10.0</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>1.2.3</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
          <?m2e execute onConfiguration?>
            <id>add-test-source</id>
            <phase>generate-test-sources</phase>
            <goals>
              <goal>add-test-source</goal>
            </goals>
            <configuration>
              <sources>
                <source>src/test/resources</source>
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.sonarsource.sonar-packaging-maven-plugin</groupId>
        <artifactId>sonar-packaging-maven-plugin</artifactId>
        <version>1.18.0.372</version>
        <extensions>true</extensions>
        <configuration>
          <pluginKey>vector-custom-java-rules</pluginKey>
          <pluginName>Vector Custom Java Rules</pluginName>
          <pluginClass>vi.sonar.rules.JavaRulesPlugin</pluginClass>
          <sonarLintSupported>true</sonarLintSupported>
          <sonarQubeMinVersion>6.7</sonarQubeMinVersion>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>

      <!-- only required to run UT - these are UT dependencies -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.1.1</version>
        <executions>
          <execution>
          <?m2e execute onConfiguration?>
            <id>copy</id>
            <phase>test-compile</phase>
            <goals>
              <goal>copy</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>org.apache.commons</groupId>
                  <artifactId>commons-collections4</artifactId>
                  <version>4.1</version>
                  <type>jar</type>
                </artifactItem>
                <artifactItem>
                  <groupId>javax</groupId>
                  <artifactId>javaee-api</artifactId>
                  <version>8.0</version>
                </artifactItem>
                <artifactItem>
                  <groupId>org.springframework</groupId>
                  <artifactId>spring-webmvc</artifactId>
                  <version>5.0.6.RELEASE</version>
                </artifactItem>
                <artifactItem>
                  <groupId>org.springframework</groupId>
                  <artifactId>spring-web</artifactId>
                  <version>5.0.6.RELEASE</version>
                </artifactItem>
                <artifactItem>
                  <groupId>org.springframework</groupId>
                  <artifactId>spring-context</artifactId>
                  <version>5.0.6.RELEASE</version>
                </artifactItem>
              </artifactItems>
              <outputDirectory>${project.build.directory}/test-jars</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

  <organization>
    <name>Vector Informatik GmbH</name>
  </organization>
  <version>1.0</version>
</project>
JavaFileCheckRegistrar
import java.util.List;

import org.sonar.plugins.java.api.CheckRegistrar;
import org.sonar.plugins.java.api.JavaCheck;
import org.sonarsource.api.sonarlint.SonarLintSide;

/**
 * Provide the "checks" (implementations of rules) classes that are going be executed during source code analysis. This class is a batch extension by
 * implementing the {@link org.sonar.plugins.java.api.CheckRegistrar} interface.
 */
@SonarLintSide
public class JavaFileCheckRegistrar implements CheckRegistrar {

  /**
   * Register the classes that will be used to instantiate checks during analysis.
   */
  @Override
  public void register(RegistrarContext registrarContext) {
    // Call to registerClassesForRepository to associate the classes with the correct repository key
    registrarContext.registerClassesForRepository(JavaRulesDefinition.REPOSITORY_KEY, checkClasses(), testCheckClasses());
  }

  /**
   * Lists all the main checks provided by the plugin
   */
  public static List<Class<? extends JavaCheck>> checkClasses() {
    return RulesList.getJavaChecks();
  }

  /**
   * Lists all the test checks provided by the plugin
   */
  public static List<Class<? extends JavaCheck>> testCheckClasses() {
    return RulesList.getJavaTestChecks();
  }
}
JavaRulesDefinition
import java.io.IOException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Locale;

import javax.annotation.Nullable;

import org.apache.commons.lang.StringUtils;
import org.sonar.api.rule.RuleStatus;
import org.sonar.api.rules.RuleType;
import org.sonar.api.server.debt.DebtRemediationFunction;
import org.sonar.api.server.rule.RulesDefinition;
import org.sonar.api.server.rule.RulesDefinitionAnnotationLoader;
import org.sonar.api.utils.AnnotationUtils;
import org.sonar.check.Cardinality;
import org.sonar.plugins.java.Java;
import org.sonar.plugins.java.api.JavaCheck;
import org.sonar.squidbridge.annotations.RuleTemplate;

import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Iterables;
import com.google.common.io.Resources;
import com.google.gson.Gson;

/**
 * Declare rule metadata in server repository of rules.
 * That allows to list the rules in the page "Rules".
 */
public class JavaRulesDefinition implements RulesDefinition {

  // don't change that because the path is hard coded in CheckVerifier
  private static final String RESOURCE_BASE_PATH = "/html"; //$NON-NLS-1$

  public static final String REPOSITORY_KEY = "vector-java"; //$NON-NLS-1$

  private final Gson gson = new Gson();

  @Override
  public void define(Context context) {
    NewRepository repository = context
      .createRepository(REPOSITORY_KEY, Java.KEY)
        .setName("Vector Repository"); //$NON-NLS-1$

    List<Class<? extends JavaCheck>> checks = RulesList.getChecks();
    new RulesDefinitionAnnotationLoader().load(repository, Iterables.toArray(checks, Class.class));

    for (Class<? extends JavaCheck> ruleClass : checks) {
      newRule(ruleClass, repository);
    }
    repository.done();
  }

  @VisibleForTesting
  protected void newRule(Class<?> ruleClass, NewRepository repository) {

    org.sonar.check.Rule ruleAnnotation = AnnotationUtils.getAnnotation(ruleClass, org.sonar.check.Rule.class);
    if (ruleAnnotation == null) {
      throw new IllegalArgumentException("No Rule annotation was found on " + ruleClass); //$NON-NLS-1$
    }
    String ruleKey = ruleAnnotation.key();
    if (StringUtils.isEmpty(ruleKey)) {
      throw new IllegalArgumentException("No key is defined in Rule annotation of " + ruleClass); //$NON-NLS-1$
    }
    NewRule rule = repository.rule(ruleKey);
    if (rule == null) {
      throw new IllegalStateException("No rule was created for " + ruleClass + " in " + repository.key()); //$NON-NLS-1$ //$NON-NLS-2$
    }
    ruleMetadata(ruleClass, rule);

    rule.setTemplate(AnnotationUtils.getAnnotation(ruleClass, RuleTemplate.class) != null);
    if (ruleAnnotation.cardinality() == Cardinality.MULTIPLE) {
      throw new IllegalArgumentException("Cardinality is not supported, use the RuleTemplate annotation instead for " + ruleClass); //$NON-NLS-1$
    }
  }

  private String ruleMetadata(Class<?> ruleClass, NewRule rule) {
    String metadataKey = rule.key();
    org.sonar.java.RspecKey rspecKeyAnnotation = AnnotationUtils.getAnnotation(ruleClass, org.sonar.java.RspecKey.class);
    if (rspecKeyAnnotation != null) {
      metadataKey = rspecKeyAnnotation.value();
      rule.setInternalKey(metadataKey);
    }
    addHtmlDescription(rule, metadataKey);
    addMetadata(rule, metadataKey);
    return metadataKey;
  }

  private void addMetadata(NewRule rule, String metadataKey) {
    URL resource = JavaRulesDefinition.class.getResource(RESOURCE_BASE_PATH + "/" + metadataKey + "_java.json"); //$NON-NLS-1$ //$NON-NLS-2$
    if (resource != null) {
      RuleMetatada metatada = gson.fromJson(readResource(resource), RuleMetatada.class);
      rule.setSeverity(metatada.defaultSeverity.toUpperCase(Locale.US));
      rule.setName(metatada.title);
      rule.addTags(metatada.tags);
      rule.setType(RuleType.valueOf(metatada.type));
      rule.setStatus(RuleStatus.valueOf(metatada.status.toUpperCase(Locale.US)));
      if (metatada.remediation != null) {
        rule.setDebtRemediationFunction(metatada.remediation.remediationFunction(rule.debtRemediationFunctions()));
        rule.setGapDescription(metatada.remediation.linearDesc);
      }
    }
  }

  private static void addHtmlDescription(NewRule rule, String metadataKey) {
    URL resource = JavaRulesDefinition.class.getResource(RESOURCE_BASE_PATH + "/" + metadataKey + "_java.html"); //$NON-NLS-1$ //$NON-NLS-2$
    if (resource != null) {
      rule.setHtmlDescription(readResource(resource));
    }
  }

  private static String readResource(URL resource) {
    try {
      return Resources.toString(resource, StandardCharsets.UTF_8);
    } catch (IOException e) {
      throw new IllegalStateException("Failed to read: " + resource, e); //$NON-NLS-1$
    }
  }

  private static class RuleMetatada {
    String title;

    String status;
    @Nullable
    Remediation remediation;

    String type;

    String[] tags;

    String defaultSeverity;
  }

  private static class Remediation {
    String func;

    String constantCost;

    String linearDesc;

    String linearOffset;

    String linearFactor;

    public DebtRemediationFunction remediationFunction(DebtRemediationFunctions drf) {
      if (func.startsWith("Constant")) { //$NON-NLS-1$
        return drf.constantPerIssue(constantCost.replace("mn", "min")); //$NON-NLS-1$ //$NON-NLS-2$
      }
      if ("Linear".equals(func)) { //$NON-NLS-1$
        return drf.linear(linearFactor.replace("mn", "min")); //$NON-NLS-1$ //$NON-NLS-2$
      }
      return drf.linearWithOffset(linearFactor.replace("mn", "min"), //$NON-NLS-1$ //$NON-NLS-2$
                                  linearOffset.replace("mn", "min")); //$NON-NLS-1$ //$NON-NLS-2$
    }
  }

}
JavaRulesPlugin
import org.sonar.api.Plugin;

/**
 * Entry point of your plugin containing your custom rules
 */
public class JavaRulesPlugin implements Plugin {

  @Override
  public void define(Context context) {

    // server extensions -> objects are instantiated during server startup
    context.addExtension(JavaRulesDefinition.class);

    // batch extensions -> objects are instantiated during code analysis
    context.addExtension(JavaFileCheckRegistrar.class);

  }

}

(Nicolas Bontoux) #11

Thanks for reporting. Have allowed .txt for now, need to check a few things before extending further.


(Tobias Gruetzmacher) #13

I have the same problem. Our internal plugin in mostly a copy of the Java Custom Rules sample (just with our own rules)…


(Michael Gumowski) #14

Hello Alex & Tobias,

I finally managed to reproduce the issue and identified the bug on SonarJava side. This is a painful regression introduced in SonarJava 5.4, which passed our review. I created the following ticket to handle it, and already started working on the fix: SONARJAVA-2805.

Note that because release of SonarJava 5.5 is a only few days ahead (end of development scheduled in 2 days), we will include the fix into that version, and not release a 5.4.1.

Thanks again for your feedback, it is priceless!

Cheers,
Michael