We’re using an on-premise SonarQube installation, and static code analysis and coverage reporting from tests is done ‘by the book’ via our build scripts. This works fine. But the thing is, we’re also running UI-Tests with Ranorex which are built on TeamCity and then executed in an on-premise installation of Azure DevOps Server (using a Release Pipeline). We’re obviously not interested in the static code analysis of our Ranorex test code as that is mostly auto-generated stuff sprinkled with tiny custom functions. The Ranorex tests however, they generate a Coverage Report (using OpenCover) which we want to send to our initially mentioned on-prem SonarQube instance.
So, excuse me rambling:
How do I send only a Coverage report to SonarQube from an Azure DevOps Server Release Pipeline?
Is there a “preferred” or “best practice” approach to this problem?
I’m going to re-state my understanding and expand a little with my guesses.
The Ranorex tests exercise your normally-built/tested/analyzed code, and you want to feed the Ranorex output into the SonarQube project for the code.
Assuming I’ve understood/extrapolated correctly, then you’ll have to run Ranorex before analysis and feed the reports in as part of analysis. It’s not possible to amend an existing analysis.
thanks a lot for the swift response. Yes, and the resulting coverage report is generated by OpenCover (which is called internally from Ranorex). So, here at the end when all went well I get an OpenCoverResult.xml file.
If I understand you correctly I can now start an Analysis without having to build my code again?
EG something like (remember, I’m in a release pipeline on Azure DevOps, so no repo access here)
-> Extract artifacts from build
-> Run Ranorex
-> Retrieve OpenCoverResult.XML
-> Do what I have not figured out yet with a Sonar Task which I have not figured out yet, too
The issue is that I can’t run Ranorex before Build, because Ranorex is running GUI Tests on the compiled artifacts from that build.
Can I send a coverage report without performing any source-code analysis? EG “Here’s my coverage report, visualize it for me please”. We would do that in a way that static code analysis would only be available in one project (I-Test Coverage from build) and the coverage from Ranorex would be reported in a second project (yellow marked in the image). Of course this only works when I can report coverage without performing static code analysis.
I think in an Azure context it’s ideally going to be something like:
Analysis start step
Build
Ranorex
Analysis end step
No. Without the underlying code, there’s nothing to visualize. Everything in SonarQube relates back to the code, so you’ll have to figure out how to include the Ranorex report as part of an analysis - whether or not you analyze the same code twice.
