Local run on AssertionsInTestsCheckTest.java failed

Please let us know what we missed.

Try to play junit assertion test by copying all the related files to local folder copied from sonar-java/docs/java-custom-rules-example, run AssertionsInTestsCheckTest.java and test failed in org.sonar.java.checks.helpers.UnitTestUtils.java:
a) line 178 for junit 4: symbol.metadata().isAnnotatedWith(ORG_JUNIT_TEST) return false for method annotated with @Test, somehow the annotation list containing Test not org.junit.Test
b) line 168 for junit 5 return false for method annotated with org.junit.jupiter.api.Test, similarly annotation list containing Test not org.junit.jupiter.api.Test
c) junit 3 also failed even Junit3Test extends TestCase
d) Changed test method annotation to @org.junit.Test in sonar-java/java-checks-test-sources/default/src/main/files/non-compiling/checks/tests/AssertionsInTestsCheck/Junit4.java working

Files copied (sonar-java version: 7.32.0-SNAPSHOT, loading files no issues, no changes for all files except file paths):

  1. sonar-java/java-checks/src/main/java/org/sonar/java/checks/tests/AssertionsInTestsCheck.java
  2. sonar-java/java-checks/src/test/java/org/sonar/java/checks/tests/AssertionsInTestsCheckTest.java
  3. sonar-java/java-checks-test-sources/default/src/main/files/non-compiling/checks/tests/AssertionsInTestsCheck/AssertJ.java
  4. files in folder sonar-java/java-checks-test-sources/default/src/test/java/checks/tests/AssertionsInTestsCheck

Thanks

Hi,

I’m struggling to understand what you’re trying to do. Could you restate, please?

 
Thx,
Ann

Try to understand juit assertions rules and then write our own junit test rule.
Question is:
By copying some junit assertion test files from sona-java to local copy of java-custom-rules-example and run for AssertionsInTestsCheck.java failed with error message “java.lang.AssertionError: No issue raised. At least one issue expected”, we expected local run should pass. Tried TestAnnotationWithExpectedExceptionCheckTest.java, failed with same error message above.
Winder if we could run these junit test classes locally? what we missed?

e.g. the following expected noncompliant test cases are all failed locally from sonar-java/java-checks-test-sources/default/src/test/java/checks/tests/AssertionsInTestsCheck/Junit4Test.java:

    @org.junit.Test
    public void containsNoAssertions() { // Noncompliant [[sc=17;ec=37]] {{Add at least one assertion to this test case.}}
    }

    static class ExtendsTestCase extends junit.framework.TestCase {
        public void testContainsNoAssertions() { // Noncompliant
        }
   }

thanks

1 Like

Hi,

Thanks for the explanation.

Not all the APIs are public, so it’s quite possible that a straight copy/paste wouldn’t work. Nonetheless, I’m not qualified to speak on that, so I’ve flagged this for the language experts.

 
Ann

Hello @ct24p,

Thanks for reaching out. For obvious reasons I can reproduce your issue, since all our unit tests are green.

Could you, please, provide a minimal isolated reproducer, so I can understand, what is the issue? (a small project I can test on my machine)

Just a random guess:

Do you provide binaries, and dependencies to the tests? The type resolution can’t work if type definitions are missing. In this case, you’ll get an “Unknown” symbol and it won’t match the desired type even if in the sources everything is correct.

Best,
Margarita

Thanks for your response. That’s exactly what i am askig— what I missed or how could i run AssertionsInTestsCheckTest.java locally?

Here is what I tried by copying the following files from sona-java (7.32.0-SNAPSHOT at github.com) to my local machine:

  1. copy all the file & structure of sonar-java/docs/java-custom-rules-example to my local machine such as: C:/java-custom-rules-example
  2. copy sonar-java/java-checks/src/main/java/org/sonar/java/checks/tests/AssertionsInTestsCheck.java to C:/java-custom-rules-example/src/main/java/org/sonar/samples/java/checks/tests/AssertionsInTestsCheck.java
  3. copy sonar-java/java-checks/src/test/java/org/sonar/java/checks/tests/AssertionsInTestsCheckTest.java to to C:/java-custom-rules-example/src/test/java/org/sonar/samples/java/checks/tests/AssertionsInTestsCheckTest.java
  4. copy sonar-java/java-checks-test-sources/default/src/main/files/non-compiling/checks/tests/AssertionsInTestsCheck/AssertJ.java, and files in folder sonar-java/java-checks-test-sources/default/src/test/java/checks/tests/AssertionsInTestsCheck/** to C:/java-custom-rules-example/src/test/files/**, such as C:/java-custom-rules-example/src/test/files/Junit4.java
  5. modify the file pathes in AssertionsInTestsCheckTest.java and run it locally as junit test (Java 17)
  6. Test run failed for all annotation @Test (junit4 or 5) or test class extends TestCase due to: line 163 of java-checks/src/main/java/org/sonar/java/checks/helpers/UnitTestUtils.java: isUnitTest(MethodTree methodTree)
  7. no runtime error
    sonar-plugin-api: 10.6.0.2114, provided scope
    sonar-java-plugin: 7.30.0.34429, provided scope

Therefore I wish to know how could I run this test class AssertionsInTestsCheckTest.java locally without failure? What parts we missed? or we could not run it locally due to type resolution can’t work?
thanks very much

Could you, please, share your project with me, as without looking at your code I can’t say what exactly is missing.

project is exactly copied from sonar-java/docs/java-custom-rules-example from github.com with modification of:
sonar-plugin-api: 10.6.0.2114, provided scope
sonar-java-plugin: 7.30.0.34429, provided scope
sonar-analyzer-commons: 2.7.0.1482
of course removed parent from pom.xml

I think I still need to know how exactly your project failed and what exactly you did in your copied version, otherwise, it might be very difficult to help.

Could you, please, push it to, for example, GitHub? Or could you at least share how is it failing (What is the error message or undesired outcome)?

Best,
Margarita

Hi Margarita:
Just created a public repo here (https://github.com/ct24p/java-custom-rules-example/tree/main), please let me know once you cloned it.
Could you please try to run java-custom-rules-example\src\test\java\org\sonar\samples\java\checks\JunitTestAssertionCheckTest.java in your local?

thanks

Thanks, I’ll have a look!

Hi:
Wonder have you get a chance to look at it? I believe the same case from here: During the execution of the rule, the obtained full name is "Test" rather than "org.junit.test"

thanks

Hey @ct24p,
Sorry for the late reply, It was vacations and conference season.

It might be indeed related so I’m adding @Leonardo_Pilastri since he’s already started the investigation.

Best,
Margarita