Importing tests from unknown languages

SonarQube Enterprise 9.8.0.63668

I’m trying to import test results for a language that SonarQube doesn’t yet understand (GoSu). I’ve got some external issues logged using generic issue data (working), and code coverage in Jacoco format (working). However, when I try to import generic test data the debug output shows something like this:

17:44:19.853 DEBUG: Skipping file 'C:\dev\path\to\a\FileTest.gs' in the generic test execution report because it doesn't have a known language


This is frustrating because both external issues and coverage work fine without SonarQube having to know what a “*.gs” file is.

The root cause of the message appears to be sonarqube/GenericTestExecutionReportParser.java at master · SonarSource/sonarqube (github.com). As far as I can see there does not appear to be any way to override this behaviour without creating a plugin for the language (why not?).

Having to create a plugin just to import generic test data involves a lot of overhead (Supporting new languages (sonarqube.org)) for something that would appear to be quite simple. And since the test output is visible on our build pipeline there will be strong resistance to committing company resources to building and maintaining it.

If there are currently no plans to be able to relax this setting, preferably on a per project bases (I’ve seen other people ask for this, e.g. GenericTestExecutionReport: Limited to known languages? - SonarQube - Sonar Community (sonarsource.com)), and without using a horrible hack… then what is the absolute minimum that needs to be in a plugin to tell SonarQube that *.gs, *.gsx, etc are GoSu files so I can get my unit tests, coverage and issues all in one place?

Hi,

On the face of it, this seems simple, so I’ve spent some time over the years (yes, it’s been years) contemplating why we don’t just handle it.

I believe this probably has to do with the larger metrics interdependency web. Consider that Coverage % is:

(conditions-evaluated-true + conditions-evaluated-false + line-coverage) / (2*condition-count + lines-to-cover)

Almost everything in that formula can be derived directly from your coverage report. But (cue ominous music) you still need lines-to-cover. And that comes from the size-related metrics that would be provided by a plugin (or native support).

So without metrics support, more than half of the coverage measures are incalculable:

“But you should just use the line counts from the coverage report!” I hear you saying. We used to. It led to inaccuracies. So we quit.

You’re going to need to provide size metrics. I’m not qualified to call out the specific metrics you need and the ones you can ignore. Admission to the Marketplace requires NCLOC & NCLOC_DATA, although I’m not certain that’s a) relevant to your question b) the minimum to get what you want. You probably need to provide executable lines data. Potentially more.

Hopefully this helps. Please do feel free to create new threads with specific development questions.

Ann

Hi Ann, thanks for the comprehensive response, super useful.

The hack I found was to force SonarQube to treat the Gosu files as the “text” type (with something like sonar.lang.patterns.text=**/*.gs,.....), but noticed that it heavily underestimated the coverage. From your post that makes sense, as I’d assume that language of course treats all lines as lines to cover even it they’re empty, comments, package imports, etc… Not an approach I’ll be progressing - inaccurate metrics are worse than no metrics. Looks like it’s build or buy.

1 Like