Importing mutation tests with PITEST

  • which versions are you using: 8.9.1 LTS
  • what are you trying to achieve: Making sure the quality of unit tests
  • what have you tried so far to achieve this: Use mutation pitests

I am checking on a Java code to see if unit tests are good and get a handle of the current situation in regards on unit tests.

I took someone else´s quality profile rules for Java and added mutation pitest rules.

Mutation tests are reporting quite poor unit tests (50% strength of unit tests).

When I try to import the pitests I get a bunch of warning saying:

WARN: Found unknown mutation operator: org.pitest.mutationtest.engine.gregor.mutators.returns.NullReturnValsMutator

or

WARN: Found unknown mutation operator: WARN: Found unknown mutation operator: org.pitest.mutationtest.engine.gregor.mutators.returns.EmptyObjectReturnValsMutator

and many similar warnings.

I run the scanner like this:

“C:\software\PackSonar\sonar-scanner-4.6.0.2311-windows\bin\sonar-scanner.bat” -D"sonar.projectKey=ProjectName" -D"sonar.sources=." -D"sonar.host.url=http://localhost:9000" -D"sonar.login=fffffffffffffffffffffffffffffffffffffffffffff"
-D"sonar.dependencyCheck.htmlReportPath=.\ProjectName\dependency-check-report.html" -D"sonar.dependencyCheck.xmlReportPath=.\ProjectName\dependency-check-report.xml" -D"sonar.coverage.jacoco.xmlReportPaths=.\ProjectName\jacoco.xml" -D"dc5.mutationAnalysis.pitest.sensor.reports.directory=.\ProjectName\target\pit-reports\202110281422"

I am wondering if there is any issue that I should be aware and if so, how to avoid it

Hi,

The Pitest plugin isn’t supported by SonarSource. Its maintainers may notice your post here, but you may have better results by posting an issue directly on that repo.

 
HTH,
Ann

Hi!

I completely understand it, but maybe you can point me in the right direction. I have asked in the forum of the mutation analysis plugin itself but no luck:

Importing mutation tests warnings · Issue #55 · devcon5io/mutation-analysis-plugin · GitHub

Apparently the pitestsensor performs some checks and can´t find the libraries mentioned

I suspect is trying to use mutators that need to be downloaded or something:

org.pitest.mutationtest.engine.gregor.mutators.returns.EmptyObjectReturnValsMutator
org.pitest.mutationtest.engine.gregor.mutators.returns.NullReturnValsMutator

I am behind a proxy, I have added to sonar-scanner.bat “-Dhttp.proxyHost=myproxy” “-Dhttp.proxyPort=8080”

I have enabled debug mode but I don´t see any specific debug info for the plugin. Is there any place where I should look at?

Thanks,

Miguel

In the end the issue was that Pitest new mutators definitions was not updated in the plugin. It has been fixed in a beta package:

Ann, what would it take to get an official Sonarqube plugin for PIT mutation testing? My company, SAP, runs a large enterprise version of Sonarqube but prefers not to install non-official plugins.
-tim

Hi Tim,

Welcome to the community!

Since this is nowhere on our priorities list, it would take a good deal of “traction”. I.e. significant support/lobbying from users.

A good first step would be to post it in Suggest new features, so it’s more visible…

 
HTH,
Ann

Thanks, Ann. I’ll do that.