Is it possible to display quality gates for pull requests only taking account the code that was added or modified in that pull request? The current “new code” approach is not serving us well. For example with a 30 day window:
Day 1: Add code with 100% coverage
Day 15: Add small amount of code with 0% coverage, total coverage is still above threshold (e.g. 90%)
Day 31: New code now has 0% coverage, but nobody even touched it. Quality gate is failing.
To be clear – the New Code Period only affects the Main Branch and any long-lived branches, when analyzing Pull Requests (seen in the Pull Requests section of your project), New Code is defined as what changed in the Pull Request. Nothing more (or less), and isn’t time based.
Sounds like we have a bug in our system then because our pull request analysis uses the same “new code” definition as long lived branches. What’s the best way to get support for private repos on paid subscriptions?
It seems that new code works correctly with the automatic detection, but it does not when the parameters are set manually. Note that comments are posted on pull requests correctly, it’s just the analysis scope that’s incorrect.
I suspect that the issue is that GitHub Actions builds the PR using the latest commit of the PR (called the PR head). However, our Jenkins jobs are configured to run the builds by merging the target branch into the PR first (called the merge head):
I kept digging and both GitHub Actions and Jenkins is building the merge head. SonarScanner can detect the merge head correctly in the repo checked out by GitHub Actions:
DEBUG: Opening pack index <repo>/.git/objects/pack/pack-1ff326c93e51b12cbf22891258fdfbdbc7e2b628.idx, size 30.540 MB took 128 ms
DEBUG: Merge base sha1: 1cdd1ff2d79c2f46df25f8e05dca188996347ea3
INFO: SCM collecting changed files in the branch (done) | time=491ms
DEBUG: SCM reported 59 files changed in the branch
But it fails to do so with the Jenkins repo:
DEBUG: Opening pack index <repo>/.git/objects/pack/pack-de250b676de0b186a1ac2cf6e2b5aadd7ac964df.idx, size 0.018 MB took 3 ms
DEBUG: Opening pack index <repo>/.git/objects/pack/pack-310c4ecd78636929c2400e41cc4b3570e05322b8.idx, size 0.247 MB took 3 ms
DEBUG: Opening pack index <repo>/.git/objects/pack/pack-e82600f5d29ad1d86ca2f40340ae0c17477601fe.idx, size 30.376 MB took 68 ms
DEBUG: Merge base sha1: 7d96b9f96d8bb763c8d4e8e6b75db58ab9fe7c89
INFO: SCM collecting changed files in the branch (done) | time=622ms
DEBUG: SCM reported 681 files changed in the branch
1cdd1ff2d79c2f46df25f8e05dca188996347ea3 is the merge base. 7d96b9f96d8bb763c8d4e8e6b75db58ab9fe7c89 is a commit from 12 days ago.
In both cases the exact same commit is checked out:
git checkout --progress --force refs/remotes/pull/14551/merge
HEAD is now at 2dcaa1d7d5e Merge c59cefbf3ba760722b473f2100e0d8cbce0e4101 into 1cdd1ff2d79c2f46df25f8e05dca188996347ea3
git log -1 --format='%H'
It appears that SonarScanner picks up an outdated local ref for the target branch instead of the up to date remote ref:
Yes, they are checking out the code differently. As I described above, the issue is that there are local references in git and remote references. The remote references are updated by git fetch, but SonarScanner first tries to find a local reference with the same name, and if that doesn’t exist then it looks for remote references.
When we use GitHub Actions, then there are no local references.
When we use Jenkins, there are some local references. The same happens in the real world on developer’s machines. Here’s a simple reproduction:
# Clone the repository, this will check out the default branch (e.g. master)
git clone repository
# Reset master to the previous commit locally
git reset --hard HEAD~1
# Check out a feature branch
git checkout origin/feature
# Run SonarScanner
dotnet sonarscanner begin \
dotnet sonarscanner end
# Notice that SonarScanner will log that "Merge base sha1" is
# where the local master branch is pointing, not the remote one