HI,
I’m using Sonarqube Developer 7.6. Recently I upgraded to Sonarqube 7.6. Looks like sonar-pitest-plugin 0.9.1 is not working with 7.6 version.
When we add pitest plugin, then sonarqube completely not even starting analysis.
Hi,
Thanks for the notification. I’ve updated the compatibility range in the Marketplace.
In fact, I believe the PiTest plugin has been abandoned. You probably want to look at the Mutation Analysis plugin (a functional equivalent) instead.
Ann
Yeah, one thing I noticed that in Quality profiles we need to activate rules of mutation.
The rule is “survived-mutant”.
With mutation-analysis-plugin we have
and for the sonar-pitest-plugin has rule survived mutant. Compared to both rules the mutation-analysis plugin is taking more time (23 mins) publish the reports in SonarQube server. Where as with pitest plugim taking 12 mins of time to publish on sonarqube server.
Looks like no one is answering. I donno why should I upgraded to Sonarqube 7.6 version.
If the sonar-pitest plugin is abandoned, we need to add one rule from sonar-pitest plugin that is “survived-mutant”. Please help me on this. I’m struck with Jenkins builds.
Hi,
Can anyone answer this issue, Please.
Hi.
I think you should ask the plugins’ developers:
Thanks Scott,
@Gerald_Mucke Can you please look into this issue we are facing.
Hi @venkat255,
I think we’ve already talked through this:
sonar-pitest-plugin is dicontinued in favor of the mutation-analysis-plugin, which is a superset of the functionality of the pitest-plugin. It’s provided as-is, if you don’t like, don’t use it. So unless you do it yourself, I doubt there will be any version of the old sonar-pitest-plugin that will be compatible with sonar 7.6
Second, this is a community/open-source offering, people like me are spending their free-time to provide code and support. There is no service-level agreement, on how fast we do answer (this is re “Looks like no one is answering”).
2 Likes
Sorry Gerald. I didn’t mean that way. I noticed in SonarQube 7.5 version using rule “survived mutant” it took less time where as the rule which is in mutation-analysis-plugin “survived mutant s” took long time.
My request is if possible please add the rule “survived mutant” to mutation-analysis-plugin.
Hi Venkat,
as written before, the rules are basically the same. The sensor parses the report, if it finds a survived mutant, it creates an issue. As simple as that. So the rule you’re requesting is the one that is implemented. And this process is rather fast (compared to the pitest analysis)
Please analyse your log, run the analysis in debug mode (-X option in Maven), and look where the time is spent. I assume most of the time is spent in running the mutation analysis (PITest) and not the Sonaranalysis.
…This rules I searched in sonarqube 7.5. Please have a look
May be I’m missing Something Gerald. The picture has survived mutant rules.
There are two rules one is “survived mutant” and the second is “survived mutants”.
“survived mutant” has test-quality
and “survived mutants” has aggregate, mutation, pitest etc.
Please see the rules in sonarqube 7.6
You missed severeal things:
- there are two rules, because one is for java and one is for kotlin. Both rules are named “Survived Mutants” and are actually the same, but each of them applies only to one language.
- “Lurking Mutants” are mutants that are not only not killed, but also not covered by any test. They are different to “Survived Mutants”, which are covered but not killed, please read the documentation,
- “test-quality”, “aggregrate”, “mutation” … are just tags so it becomes easier to find the tags in the rules repository, nothing else
- all of these rules are marked deprecated. I advise you to switch to the mutation-operator specific rules instead as these provide more detailed information about the mutations. Don’t mix the generic, deprecated rules and the mutation-specific rules as it would double the number of issues without any added value. Again: read the documentation.
- you still missed to provide logs or any other information to prove your claim, that these rules now take twice as long as before. While it might be possible, I have doubts, as from my experience the pitest-analysis takes orders of magnitude longer than parsing the reports. Further, the sonar analysis itself also takes time, and may behave differently in the new release. Without any evidence, there is nothing anyone could do to analyse the problem or provide a fix of any kind - if applicable at all.
Sorry, I didn’t expain clearly in picture. Yeah, Im using for JAVA only. I didn’t post right picture.
SonarQube 7.5 with rules
SonarQube 7.6 with rules. here we missed the rule survived mutant which was not deprecated one. I’m trying to post log files but not able to post logs. I hope this will help to understand.
I know we all working for efficient quality analysis.
Alright, you remember that you removed the old sonar-pitest-plugin (0.9.1) that was not working in Sonarqube 7.6? That was the plugin that provided the “Survived mutant” rule, without any tags and without deprecation. If you check the rule, you’ll see it’s from a different repository.
Inside, the old rule from the old plugin, and the deprecated rules from the new plugin, work the same: if there is a survived mutant, then create an issue. Now that you don’t have the old sonar-pitest-plugin anymore, you don’t have this rule anymore. You have to adapt your rules profile that you’re using to use the deprecated rule - or better - use the mutation-operator specific rules.
- sonar-pitest-plugin (0.9.1)
- not compatible with Sonarqube 7.6
- provides “Survived Mutant” rule (only “test-quality” tag, no deprecation)
- mutation-analysis-plugin (1.4)
- compatible with Sonarqube 7.6
- provides “Survived Mutants” rule (has more tags, is deprecated)
- provides Mutation-operator specific rules (has more tags, a better description about the mutation, is not deprecated) <— USE THESE
All of these are applied to the parsed xml report (mutations.xml) that is produced by PITest.
For each of the surviving mutants in the report, an issue is created - if the corresponding rule is active.
PITest takes very long to compute and requires a lot processing power.
Parsing the report doesn’t take long.
Does this make things a bit more clear to you?
Thanks Gerald. Sorry for the late reply.
I’m 100% clear on the above explanation. I saw the rules both plugins.
Now I run the analysis with Mutation-analysis-plugin (1.4) with only one mutation-operator rule. It took analysis time 24 mins where as I Ran again with all mutation-operator rules still it took 24 mins. I agree this rule set has more aggregate and rules that’s y issues also increased. I tested all possible ways Gerald.
Now, here the point, I analysed with sonar-pitest (0.9.1) on SonarQube 7.5 then it publish the report on sonarqube in 12 mins with “Survived mutant” rule (only “test-quality” tag, no deprecation). Here, our planning to publish pit reports to SonarQube dashboard.
Is there any possible way can we add the “Survived mutant” rule (only “test-quality” tag, no deprecation) to mutation-analysis-plugin (1.4). So that we can use any of the rules what ever we need. May be I’m wrong but honestly I tried all the possible ways Gerald.
Hi Venkat,
Adding the rule you’re asking for will not solve any problem unless we understand, what the actual problem is. Asking for adding a rule, is like asking for changing sunglasses to drive faster after having exchanged car and driver.
To give some explanation: the rules don’t do anything, the code is basically “if rule is active, and a condition is met for this rule, create an issue”, adding a new rule with old name would change this to “if new rule with old name is active, and a condition for this rule is met, create an issue”.
Nevertheless, there could be parts in the code - that are not related to a specific rule - that could be optimized. But I didn’t had any performance issues so far, therefore I didn’t profile the plugin or run any benchmarks, as this would require some time, which I honestly don’t have a lot to spare.
So unless you provide a detailed log of your build/analysis execution, do a proper performance analyis or provide a proper reproducer, that in someway would help to pinpoint where the actual problem is, nothing is going to happen.
Here are some ideas to spark your creativity in problem search:
- run an analysis on sonarqube 7.6 without any mutation analysis rules active, but with running the pitest analysis (the maven/gradle plugin)
- activate more or less mutation-analysis-rules
- analyse logs / debug logs, with timestamps, to see when mutation-analysis-plugin starts/finishes
- run just the pitest analysis (no sonarqube at all)
- what are the jvm settings of your build
- what is the hardware configuration of your build system
But in summary: provide some solid proof, that the performance problem is caused by the mutation-analysis-plugin.
Thanks Gerald. May be I’m missing something. Can you please suggest how can I share the logs files.
I’ll send both logs with activating one rule and activating all rules in mutation-analysis-plugin
Hi Venkat,
please open an issue in the github project https://github.com/devcon5io/mutation-analysis-plugin/issues/ with a summary description of the problem and a link to this forum thread. You can attach files to the issue. As the files are public, please ensure the logs contain no confidential information.
Please ensure gradle (or whatever build tool you’re using) has the debug-log level enabled (in maven it’s the -X flag). And if possible, also include the mutations.xml produced by pitest, so I could re-run the analysis itself for analysis.
Hi Gerald,
I tried all possible ways to share logs but not able to share all the logs. I analysed with time stamp. This project has 10 modules and each module has mutations.xml file. When the mutation-analysis sensor running on module it is taking long time with one rule active.
Please find the attached log. Hope this will help. Looks like instead of parsing mutations.xml file it is doing mutation analysis. If I’m not wrong. Here only it is taking extra of 12 mins. I activated one rule in mutation-rules.
14:35:24 1587/1587 source files have been analyzed
14:35:24 Classes not found during the analysis : [javax.el.ELContext]
14:35:24 Java Main Files AST scan (done) | time=285288ms
14:35:24 Java Test Files AST scan
14:35:24 101 source files to be analyzed
14:35:32 101/101 source files have been analyzed
14:35:32 Java Test Files AST scan (done) | time=8189ms
14:35:32 Sensor JavaSquidSensor [java] (done) | time=293637ms
14:35:32 Sensor Mutation Analysis [mutationanalysis]
14:35:32 Pitest Sensor [java] running on [key=project:xyzDotCom:xyz.xyzDotCom.module] in E:\Jenkins\workspace\ci_pipeline\xyzDotCom\xyz.xyzDotCom.module
14:47:24 Sensor Mutation Analysis [mutationanalysis] (done) | time=698652ms
14:47:24 Sensor SurefireSensor [java]
14:47:24 parsing [E:\Jenkins\workspace\ci_pipeline\xyzDotCom\xyz.xyzDotCom.Core\build\test-results\test]
14:47:24 Sensor SurefireSensor [java] (done) | time=70ms
14:47:24 Sensor JaCoCoSensor [java]
14:47:24 Analysing E:\Jenkins\workspace\ci_pipeline\xyzDotCom\xyz.xyzDotCom.Core\build\jacoco\test.exec
Hi Venkat,
alright, this log clearly indicates the mutation analysis takes ~12 mins
14:35:32 Pitest Sensor [java] running on [key=project:xyzDotCom:xyz.xyzDotCom.module] in E:\Jenkins\workspace\ci_pipeline\xyzDotCom\xyz.xyzDotCom.module
14:47:24 Sensor Mutation Analysis [mutationanalysis] (done) | time=698652ms
What should I do next? What would you do to narrow down the problem?