[Java] Class members annotated with @TestOnly should not be accessed from production code

Clean Code New rules / language support
1

Description
JetBrains published a set of annotation that can be interpreted by IDEs and static analysis tools to improve code analysis. The annotation @ TestOnly can be used to mark methods, constructors, fields and types whose visibility restrictions has been relaxed more than necessary for the API to allow for easier unit testing.

Access to such methods is fine for code in test packages but is discouraged in production code. It would be nice if there was a Sonar rule that could raise an issue if the latter case is encountered, so violations could be spotted quickly in SonarQube and would also be flagged in the IDE by SonarLint.

The rule should cover not only fields but also methods, for scenarios where using a getter is fine in production code but a setter is only allowed to be used in test code.

Noncompliant Code:

/** src/main/java/MyObject.java */
@VisibleForTesting
String foo;

/** src/main/java/Service.java */
new MyObject().foo; // NONCOMPLIANT, foo is accessed from production code

Compilant Code

/** src/main/java/MyObject.java */
@VisibleForTesting
String foo;

/** src/test/java/MyObjectTest.java */
new MyObject().foo; // COMPLIANT, foo is accessed from test code

External references

Github repo:

Sonar is already acknowledging the existence of @ VisibleForTesting as exceptions to RSPEC-5803. Maybe this one can be extended or a new one can be created for @ TestOnly.

Type
Code Smell - It’s meant to raise awareness to potential violations of API contracts. Ideally the original developer recognizes the issue right there in the IDE and refrains from accessing that member in that way.

Tags
pitfall - Modifying the internal state of objects in a manner that is not sanctioned by the API in production code could cause unforeseeable problems further down the line and might happen easily by accident if a developer does not check the JavaDoc or original member definition to notice that annotation before using it.

2

Your copy-pasted rule description seems to get the purpose of this annotation wrong. JetBrains actually also has an annotation named @VisibleForTesting in the very same package as the @TestOnly annotation you linked.

The difference is that @VisibleForTesting is okay to be called from production code if it’s possible to do so when the visibility of the member would be stricter than it actually is for testing purposes, whereas @TestOnly should never be called from anywhere except test code. So just adding it to the list of annotations from the existing rule would not be enough. It would kinda need to be its own rule to make sense.

5

Hello,
we recently added support for cases when the @VisibleForTesting annotation has some parameters in the case of androidx.annotation, since you can specify which level of visibility the field/method would have had without the annotation.

I feel like this is another edge case of the same rule, this case being more strict.
I created a ticket to support this, specifying that members marked with this annotation should never be referenced from production code, even within the same file.