How can we measure reliable coverage in SonarQube Cloud?

SonarQube Cloud
1

Hello everyone, I am new to the community.

I apologize in advance if I am not following any forum guidelines. I am reaching out to ask for your guidance regarding code coverage in SonarQube Cloud.

We use GitHub as our source code repository, GitHub Actions to automate our CI/CD pipelines, and SonarQube Cloud Enterprise as our code quality platform.

We are currently implementing coverage reporting for Java, JavaScript, and TypeScript projects using tools such as JaCoCo and Jest. The reports are generated in our pipelines and then imported into SonarQube Cloud.

Our main concern is that a high coverage percentage confirms that the code was executed during testing, but it does not necessarily mean that the tests are valuable or contain meaningful assertions. This could encourage teams to create tests only to meet the Quality Gate requirements.

The governance of the platform is managed by a department outside the development teams. Therefore, we do not participate directly in the development process or have detailed knowledge of the functional behavior of each project.

Given this context, we are looking for the best way to assess code quality using SonarQube Cloud Enterprise.

We would appreciate your guidance on the following questions:

  • What alternatives are available for measuring coverage besides unit tests? Is it possible to measure coverage without relying exclusively on them?
  • If no alternative is possible, what would be the best way to implement unit tests to prevent low-value tests from distorting the coverage metric?
  • What additional controls or metrics would you recommend for establishing a reliable and automated Quality Gate?

Thank you very much for your guidance :smiley:

2

Hey @Gabriel182, welcome to the Community! You are following the forum guidelines quite well, worry not :grin: I’ll go over your 3 questions:

  • What alternatives are available for measuring coverage besides unit tests? Is it possible to measure coverage without relying exclusively on them?

You can import coverage generated by other automated test types (not only unit tests) into SonarQube, provided the selected coverage tool produces a compatible report. However, coverage from both sources will be merged additively, so this won’t discount the coverage reported by other weak unit tests. I wouldn’t recommend pursuing this.

  • If no alternative is possible, what would be the best way to implement unit tests to prevent low-value tests from distorting the coverage metric?

The best thing you can do is ensure that SonarQube is analyzing your test code in a way that will raise issues when unit tests are low-quality. Ensure test directories are correctly classified and analyzed, then activate the assertion-related test rules for Java and JavaScript/TypeScript (if they are currently disabled). Normally, low quality test code should raise some issues.

  • What additional controls or metrics would you recommend for establishing a reliable and automated Quality Gate?

Closely linked to the previous point. You should make sure that issues raised from these test/assertion-related rules will break the Quality Gate. For example, you could raise the relevant rule’s severity to blocker and add a Quality Gate condition of Blocker Issues > 0 on new code.