Coverage threshold ok during PR build but failing after merge

Hi,
This issue seems to be related with:

But they are a bit old and I couldn’t understand if there’s a way to avoid this entirely by now.

Info:

  • ALM used: GitHub
  • CI system used: Github Actions
  • Scanner command used:
    Run SonarSource/sonarqube-scan-action@v4.2.1
    with:
      projectBaseDir: ./
      args: -Dsonar.organization=mov-ai 
            -Dsonar.projectKey=MOV-AI_movai-core-shared 
            -Dsonar.sources=. 
            -Dsonar.scm.provider=git 
            -Dsonar.qualitygate.wait=true 
            -Dsonar.qualitygate.timeout=300 
            -Dsonar.coverage.exclusions=tests/**,docs/** 
            -Dsonar.cpd.exclusions=tests/**,docs/** 
            -Dsonar.python.version=3 
            -Dsonar.python.flake8.reportPaths=flake8-report.txt 
            -Dsonar.python.pylint.reportPaths=pylint-report.txt 
            -Dsonar.python.coverage.reportPaths=coverage.xml
    
  • Languages of the repository: Python
  • URLs provided above
  • Error observed: coverage ok on PR but fails after merge
  • Workaround: Lower temporarily the QualityGate :frowning:
  • Quality gate:
  • PR result:
  • Branch result:

As mentioned we’ve been working around this issue by temporarily lowering the Quality Gate.
What can we do as a permanent solution?

Thanks

Hi,

Let’s take a closer look at your first screenshot:

It looks to me like it was already broken before this PR.

 
Ann

Thanks for pointing that out, but despite that the verification succeeded. Should it have failed then?

Hi,

Your Quality Gate condition is coverage on new code. That was 80.6%.

 
Ann

Right, that would indicate what is considered new code in the branch analysis is different from what I expected (the differences introduced in the PR). Is there a way to configure this behavior?

Hi,

Yes, that’s exactly right. What’s considered new in the PR is what changed in the PR. What’s new in a branch is based on the branch’s new code definition.

 
HTH,
Ann