Java Parser failure but no issue and lost scan with SonarJava 6.0.0-SNAPSHOT

  • Versions
    SonarQube 7.9.1 LTS, SonarJava 6.0.0-SNAPSHOT (seen over a month ago, but just verified commit 80a0ee431b4b04004613fd76e3084f24fb141264 from 9/23/2019)

  • Error observed
    When there is a parser error, at least this specific parser error as listed below, there is no Issue from the scan (the Parser Check does not get a chance to check it.) Instead the scan for the problem Java file is omitted and lost. Instead the JavaAST, but the error below shows up in the Logs.

[ERROR] Unable to parse source file : 'src/test/java/test/AppTest.java'
[ERROR] Parse error at line 3 column 42: Syntax error on token ";", delete this token
  • Steps to Reproduce
    This is possible to introduce just by putting in an extra semicolon at the end of an import statement that is not the last import statement. I can’t upload a java file, so here’s the full file in the markdown…
package test;

import static org.junit.Assert.assertTrue;;
import static org.junit.Assert.assertFalse;;
import org.junit.Test;

public class AppTest 
{
    @Test
    public void shouldAnswerWithTrue()
    {
        assertTrue(true);
        assertFalse(false);
    }
}

As a result of the ERROR, the code is not parsed and not scanned for issues, and it shows up in SonarQube in the Code tab without any code markup or color. In the 5.13 and 5.14 sonar-java-plugin, this error was not reported in the logs, and the invalid java syntax was ignored. It probably should be stated that it is indeed a valid parser problem which the JDK also ignores, and is still listed as an open bug in the JDK. https://bugs.openjdk.java.net/browse/JDK-8027682

  • Potential Workarounds
    – Use the 5.14 plugin.
    – Ignore the error and go without the scan of the affected file.
    – Add automated log scanners in the Continuous Integration system to call out the issues so they can be fixed, and then fix them so all issues in the problem files can be scanned.

Hello @hajush,

Thank you for reaching out. Like you probably know, version 6.0.0-SNAPSHOT of SonarJava is still under (heavy) development, and absolutely not production ready. Reporting on it is consequently a bit premature.

To give you more insight, we are changing the core of the SonarJava plugin from our own frontend to another one. This will be a major change, hence the decision to do a major release 6.0 at the end of the process.

Now, we are aware of this particular issue (and many other one we still need to deal with) and are still discussing about how/if to address it.

Our previous parser was allowing extra semicolons inside imports, and was therefore behaving erroneously, by allowing what a bug from JDK was allowing. This is already quite controversial. Are we supposed to reproduce bugs from JDK itself? The Java Language Specification is quite explicit about where semicolons are allowed in compilation units. See JLS 13 - §7.3 Compilation Units, where it is said that semicolons are allowed only in the type declarations, not in the imports (emphasis mine).

Extra “;” tokens appearing at the level of type declarations in a compilation unit have no effect on the meaning of the compilation unit. Stray semicolons are permitted in the Java programming language solely as a concession to C++ programmers who are used to placing " ; " after a class declaration. They should not be used in new Java code.

The new parser we are using is following exactly the JLS on this, and therefore does not handle what is considered as invalid java code.

Regarding the potential workarounds:

We would discourage to use a 6.0 snapshot version of the plugin, so staying for the moment with the latest release 5.14;

Should be the default and actual behavior of the plugin. We might have slightly changed it, but should not be the case.

Rule squid:ParsingError, which is not enabled by default, should do exactly that, and report files which failed parsing.

4 Likes