After upgrade to 9.9 LTS, custom rules keys are all set to "external.catchall"

We are facing an important issue with the update from 8.9 to 9.9 LTS, specifically with the reporting of our custom Detekt & Android Lint rules to Sonar. For context, we are using Sonar on an Android project (thus using the sonar-kotlin plugin as well as the external rule engines Android Lint and Detekt).

By custom, we mean rules that are not part of the official Detekt and Android Lint repositories. For instance, we are using rather popular external rules such as Twitter Jetpack Compose Rules, Formatting Rule Set | detekt as well as some Android Lint rules we wrote ourselves.

Here is our workflow:

The CI launches the detekt, lint, then sonar analysis. This uploads the reports automatically to SonarQube. Then, a second job will fetch sonar issues via the Web API to upload them to an external board. We extensively use that board to filter our code smells per issue type (DeprecatedUsage, UnusedImport, etc.).

Previously, with the 8.9 version, when having custom rules, they were categorized using their defined issue type id we defined when creating those rules. That allowed us to fetch them through the API to put them into our external board, on which we were able to filter by IssueType (e.g. BadImport, QontoDeprecation, WrongComposeParameters, etc.), without any other specific work.

Starting with Sonar 9.9 LTS, we have seen all those custom issues regrouped into one big external.catchall issue-type bag per external tool. Because of this, we are no longer able to filter/group them by specific type.

This is not only an issue on our external board but also on the SonarQube Web UI:

image

Instead of seeing a breakdown of issues grouped by specific Rule (like LongMethod and UnusedImports here) we now have two big “Android Lint Rule” and “Detekt Rule” categories that each contain 400+ issues that are technically coming from all our different custom rules.

We have thought about several possibilities to work around this:

  1. From what we understand, the change that triggers this new behavior is inside the sonar-kotlin plugin. There are now rules.json files that contain a list of all the official Detekt and Android Lint rules, if an issue is reported with a rule key that is not present in those files it’s marked as a “catchall” type. One possibility would be for us to fork the project and add our custom rules inside it. However, this doesn’t seem like a long-term solution (too much maintenance)

  2. Customize the ‘message’ of our custom issues to add the issue type in it, then have our ‘Sonar to Board’ script parse the messages to extract the information. However, while it allows us to have the right category in our external boards, this does not fix the categorization on SonarQube’s interface and it’s rather hacky

  3. Implement a custom issue reporter. Instead of providing sonar with detekt & lint reports, we would have to parse those internally, and output a ‘generic issue report’. Out of the 3 workarounds, this seems to be our best bet, but it’s time-consuming and something we would have to maintain. We would basically have to duplicate the logic of what sonar-kotlin is doing and override that “catchall” logic so it picks the rule key instead.

All of those workarounds sound hacky and/or time-consuming. It appears the issue originates within the sonar-kotlin plugin. Could Sonar possibly address this problem directly and not throw issues with custom rules in a “catchall” rule? We’re open to discussing this with the development team if there’s any confusion or if additional details are required.

Is there another solution we haven’t considered yet?
Additionally, was this issue documented anywhere? We reviewed the SonarQube release notes before upgrading but found no mention of it.

Thank you

Hey there.

Your description runs counter to how we expected the import of external issues to work in 8.9.

I want to try and reproduce this but didn’t get to it before the end of this week. Thanks for your patience!

Hey there.

I’m really sorry for the very late reply on this one.

I tried it out – and you’re right, in SonarQube v8.9 LTA custom rules of external linters (at least Detekt) were correctly imported into SonarQube.

I just edited the example here (after running ./run-external-linters.sh) to some custom rule keys.

SonarQube 8.9 LTA

<?xml version="1.0" encoding="utf-8"?>
<checkstyle version="4.3">
<file name="src/kt/android-dagger/app/src/main/java/com/example/dagger/kotlin/ApplicationComponent.kt">
	<error line="28" column="1" severity="warning" message="The file ApplicationComponent.kt is not ending with a new line." source="detekt.Custom" />
</file>
<file name="src/kt/android-dagger/app/src/main/java/com/example/dagger/kotlin/DemoApplication.kt">
	<error line="28" column="1" severity="warning" message="The file DemoApplication.kt is not ending with a new line." source="detekt.OtherCustom" />
</file>
</checkstyle>

And that same report in SonarQube v9.9 LTA (and SonarQube v10.5 for that matter) will be collapsed into the catch-all rule.

I think what happened is that we used to filter on the prefix of rule key to see if we should import it (for Java and Checkstyle this was com.puppycrawl.tools.checkstyle.checks.. For detekt it was just detekt.).

I’m not sure why we decided that to support external linters we needed to create a catch-all rule, and not just remove the prefix check. :thinking: Maybe something went really wrong when that prefix was different than expected. And to find that out I have to throw it back to the team.

Sorry again for the delay.

Cheers Colin, thanks for the response!
In the meantime, we’ve created our own generic-issue-report for all our external tools as mentioned in Option 3 but we would still love for Sonar to publish a fix to revert to the old behavior.
This would prevent us from having to maintain custom scripts and would also look nicer on the SonarQube UI (the catch-all rule means we’re losing the rule description for instance).

Hello @FloQonto,

Thanks for reporting this issue. I can see what led to this change. And I created the ticket to fix it in future releases:

https://sonarsource.atlassian.net/browse/SONARKT-387

Just a side question, were you able to see the rules description of those rules in SonarQube in 8.9?

Best,
Margarita

Cheers Margarita! I’m pretty sure we did have the description, as we’ve migrated I can’t rely check easily but I believe @Colin should be able to double check on its instance.

1 Like

Thanks, I will see what we can do from our side.