Pull request analysis displays "0 new lines" for all PRs and shows no coverage information

[vicsonvictor]

Version used: sonar-scanner-cli-4.7.0.2747
Trying to achieve: Get coverage information displayed on SonarCloud and Bitbucket Cloud Pull Request quality gate.
Things tried so far: Followed the documentation to set up pull request analysis; Turned on DEBUG logs to print details during processing; Tried some other ideas shared on other related topics.

We have an android multi module project on bitbucketcloud. We use Bitrise for our CI. We generate the coverage reports using Jacoco(4 different ones for the individual modules) and push them up to SonarCloud using the scanner step on Bitrise(4 xml reports).

I can see that in the logs that the 4 reports are picked up. And I see an “estimated coverage after merge” on the portal. However, it always says “0 new lines” on the PR and does not display coverage information on the portal and also on the quality gate on the PR on Bitbucket.

I see this line in the logs: “SCM reported 0 files changed in the branch” and “SCM reported changed lines for 0 files in the branch”, so looks like the diff is not recognized.

As suggested on another topic, I ran the following command in a step prior to the Scanner step on Bitrise and see the diffs I expect to see: git diff --name-only origin/master…HEAD

Here is my sonar config that I am passing to the Scanner CLI:

sonar.projectKey=[REDACTED]
sonar.login=[REDACTED]
sonar.organization=[REDACTED]
sonar.host.url=https://sonarcloud.io
sonar.sourceEncoding=UTF-8
sonar.projectName=[REDACTED]
sonar.language=kotlin
sonar.log.level=DEBUG
sonar.sources=provider/src/main/java,provider/onboarding/src/main/java,uiToolkit/src/main/java,networking/src/main/java
sonar.tests=/bitrise/src/provider/src/test/java,/bitrise/src/provider/src/[REDACTED]Test/java,/bitrise/src/provider/onboarding/src/test/java,/bitrise/src/provider/onboarding/src/[REDACTED]Test/java,/bitrise/src/uiToolkit/src/test/java,/bitrise/src/uiToolkit/src/[REDACTED]Test/java,/bitrise/src/networking/src/test/java,/bitrise/src/networking/src/[REDACTED]Test/java
sonar.pullrequest.provider=bitbucketcloud
sonar.pullrequest.branch=PRO-1142
sonar.pullrequest.base=master
sonar.pullrequest.key=658
sonar.pullrequest.bitbucketcloud.owner=[REDACTED]
sonar.pullrequest.bitbucketcloud.repository=[REDACTED]-[REDACTED]
sonar.coverage.jacoco.xmlReportPaths=/bitrise/src/provider/build/reports/jacoco/jacoco.xml,/bitrise/deploy/jacoco_reports/*.xml

(“provider”, “provider/onboarding”, “networking”, “uiToolkit” are the 4 different modules we have directly under the root of the project, hence the reference on the sources parameter.)

What am I missing?

I am stuck with this issue and am unable to progress with Sonar. Your help is highly appreciated. Thanks.

[vicsonvictor]

If it helps, I can share the relevant logs generated on Bitrise for the scanner steps. There is a lot of information in there, not sure what is relevant to the issue I am facing. Thanks.

[vicsonvictor]

The WARN lines I see in the logs, not sure if this is the root cause:

02:16:18.997 WARN: Dependencies/libraries were not provided for analysis of TEST files. The ‘sonar.java.test.libraries’ property is empty. Verify your configuration, as you might end up with less precise results.
02:16:19.014 WARN: Unresolved imports/types have been detected during analysis. Enable DEBUG mode to see them.
02:16:19.023 DEBUG: Unresolved imports/types:

• Description cannot be resolved to a type
• Matcher cannot be resolved to a type
• RecyclerView cannot be resolved to a type
• Resources cannot be resolved to a type
• The import [REDACTED] cannot be resolved
• The import [REDACTED]x cannot be resolved
• The import org.hamcrest cannot be resolved
• TypeSafeMatcher cannot be resolved to a type
• View cannot be resolved to a type

02:16:20.828 WARN: Access to the multi-values/property set property ‘sonar.java.binaries’ should be made using ‘getStringArray’ method. The SonarQube plugin using this property should be updated.
02:16:20.828 WARN: Access to the multi-values/property set property ‘sonar.java.libraries’ should be made using ‘getStringArray’ method. The SonarQube plugin using this property should be updated.

[ganncamp (G Ann Campbell)]

Hi,

Since you’re working in Kotlin, does that mean you’re building with Gradle? If so, I think a first step would be to switch to the SonarScanner for Gradle. At a minimum, it would significantly reduce your analysis configuration.

And looking at your configuration, I notice some of your paths start with /, making them absolute from box root. I suspect that’s part of the problem here.

Could you share your analysis logs?

 
Ann

[vicsonvictor]

@ganncamp Thanks for taking a look. We are using gradle for the different steps in our CI(Bitrise) workflow. We are using the sonarqube scanner step available on Bitrise to push the coverage reports to SonarCloud. Please find the logs attached, it also prints the sonar parameters configured for the PR analysis. Tried a bunch of other updates including source and binaries path. Nothing seems to make an impact.
apps_b100f922f79ced7c_6191d781-8022-4b5d-bb60-9af1009989fa-full.txt (5.8 MB)

Please search for “id: sonarqube-scanner” to locate the logs for the sonar step in the attached file.

[vicsonvictor]

@ganncamp BTW, the sonar gradle plugin is not an option for us due to this reported issue.

Sonarqube gradle plugin task not available on dependency sub module

Must-share information (formatted with Markdown): which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension) SonarQube Gradle plugin: v3.3 what are you trying to achieve Make the sonarqube gradle task run for a nested android module, which lives under another android module in the project repository structure. Due to the way our CI is setup. We need to run a separate workflow for all our different android modules with tests and then push up the coverage data t…

[vicsonvictor]

@ganncamp Did you get a chance to look at the log file I had attached? Thanks.

[ganncamp (G Ann Campbell)]

Hi,

Are there changes / additions to actual code files in the PR you sent the analysis log for? I’m seeing this at the top of the file:

bitrise.yml                                        |   2 +
build.gradle                                       |   2 +-
gradle-scripts/jacoco_module.gradle                |   2 +-
.../shift/highlights/ShiftHighlightsScreenTest.kt  | 203 ---------------------
4 files changed, 4 insertions(+), 205 deletions(-)

I recognize that the deletions were changes in a .kt file, but there’s no way to annotate changes on deleted lines.

Also, I notice a number of problems parsing your coverage report. E.G.

15:05:29.987 ERROR: Cannot import coverage information for file 'uiToolkit/src/main/java/com/shiftkey/uiToolkit/common/ui/compose/Snackbar.kt', coverage data is invalid. Error: {}
java.lang.IllegalStateException: Line 65535 is out of range in the file uiToolkit/src/main/java/com/shiftkey/uiToolkit/common/ui/compose/Snackbar.kt (lines: 63)

And finally, sonar.language=kotlin has no effect. You can remove it. Similarly, where did you find sonar.kotlin.binaries? I don’t see it in the docs, and it’s not clear to me that it does anything.

 
Ann

[vicsonvictor]

Thanks for responding Ann.

On that particular PR, there were no changes to the source. But I removed the test class and that should have resulted in a change in coverage data.

But here is a recent one:

https://sonarcloud.io/summary/new_code?id=shiftkeyllc_shiftkey-android&pullRequest=698

Printing git diff results...

domain/src/main/java/com/shiftkey/provider/repository/AccountRepository.kt

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testInvalidEmailError.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testNormalState.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testRequestFailedState.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testRequestInProgressState.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testSaveButtonVisible.png

provider/src/[REDACTED]Test/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailScreenTest.kt

provider/src/main/java/com/shiftkey/provider/data/repository/account/AccountRepositoryImpl.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/PersonalInfoFragment.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailFragment.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailScreen.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailViewModel.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/phone/ChangePhoneScreen.kt

provider/src/main/res/navigation/navigation_profile_signed.xml

provider/src/test/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailViewModelTest.kt

uiToolkit/src/main/java/com/shiftkey/uiToolkit/common/ui/compose/form/FormValidators.kt

I am attaching the full log for that CI run with this message.

apps_b100f922f79ced7c_84afe71c-e4e4-4110-b2de-cfec027e37a9-full.txt (6.3 MB)

The parsing problems you have pointed out are due to the presence of some expanded kotlin files with inline functions. I am assuming that those individual errors are not the reason for the “0 new lines” issue.

I will remove the properties that you have pointed out as irrelevant.

[ganncamp (G Ann Campbell)]

Hi,

Regarding your previous log,

@vicsonvictor

On that particular PR, there were no changes to the source. But I removed the test class and that should have resulted in a change in coverage data.

PR analysis reports on changes to source code. Removing a test isn’t going to show up unless the (previously) tested code is changed in the PR.

Regarding your new one… am I right in thinking you’re doing a merge commit before analysis? That might explain why you’re getting “SCM reported 0 files changed in the branch”. PR analysis isn’t designed to work on merge commits. You need to give it the plain PR, not what it would look like after merge back into master.

 
Ann

[vicsonvictor]

Thanks Ann. On the new one, no, we are not doing a merge commit before analysis. On bitbucket, we see the exact set of files listed above using the “git diff --name-only origin/master…HEAD” command.

Before we execute the sonarqube scanner step in our CI workflow, I have a shell script with the above git command to print the diffs between the src and dest branch.

The pull request points to master and the diff lists out the difference between head of the source branch to the destination branch which is the master.

[vicsonvictor]

In fact, you can look any of our PRs at https://sonarcloud.io/project/pull_requests_list?id=shiftkeyllc_shiftkey-android and you will see most of them pointing to master and none of them have a merge commit on the source branch.

[vicsonvictor]

@ganncamp Thanks for looking into this issue for us. I have been evaluating Sonar for weeks now and this single issue is blocking me from arriving at a conclusion. If at all possible, can we please jump on a conference call(zoom), so I can show you all the different pieces - bitbucket, bitrise(CI) and Sonar.

[ganncamp (G Ann Campbell)]

Hi,

I’m not finding anything interesting/relevant in the analysis log.

So I have 2 questions:

• what happens in this branch/PR? Are there actual changes to lines of code?
• how exactly are you checking out the branch? Because I’m noticing this: git "checkout" "--detach"

 
Ann

[vicsonvictor]

1. Yes, this PR has actual changes as you can see from the diff list before(which is in the logs):

Printing git diff results...

domain/src/main/java/com/shiftkey/provider/repository/AccountRepository.kt

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testInvalidEmailError.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testNormalState.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testRequestFailedState.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testRequestInProgressState.png

provider/src/[REDACTED]Test/assets/screenshots/email.ChangeEmailScreenTest_testSaveButtonVisible.png

provider/src/[REDACTED]Test/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailScreenTest.kt

provider/src/main/java/com/shiftkey/provider/data/repository/account/AccountRepositoryImpl.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/PersonalInfoFragment.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailFragment.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailScreen.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailViewModel.kt

provider/src/main/java/com/shiftkey/provider/ui/profile/signed/options/personal/phone/ChangePhoneScreen.kt

provider/src/main/res/navigation/navigation_profile_signed.xml

provider/src/test/java/com/shiftkey/provider/ui/profile/signed/options/personal/email/ChangeEmailViewModelTest.kt

uiToolkit/src/main/java/com/shiftkey/uiToolkit/common/ui/compose/form/FormValidators.kt

2. Yes, the “git clone” step in our CI workflow executes the git “checkout” commands to checkout both the source and destination branches.

[ganncamp (G Ann Campbell)]

Hi,

@vicsonvictor

Yes, this PR has actual changes as you can see from the diff list before(which is in the logs):

Yes, I saw those log lines. There are changes in 4 non-test source files. What I’m trying to verify is that there are changes in one or more of those files to source code, versus e.g. comments or whitespace.

@vicsonvictor

executes the git “checkout” commands to checkout both the source and destination branches.

Okay, new code detection relies on your SCM data. You’ve just said you’re checking out two branches and I see from your log that you’re doing a --detach. Frankly, I have no idea what impact that has, if any. But it’s not a plain checkout. Can you try this with just a vanilla checkout of the PR’s branch?

 
Ann

[vicsonvictor]

Actually, there are 10 kotlin files in that list which have non-comment/whitespace changes - actual code changes.

I will try to remove the “–detach” option in the checkout command. Not sure if I have that control in the git-clone step on bitrise.

Another question for you, I see these lines in the logs. Do any of them offer any clues?

14:21:08.074 INFO: SCM collecting changed files in the branch
14:21:08.089 DEBUG: file=/bitrise/src/.git/config, create new FileSnapshot: lastRead=2022-04-27 14:21:08.087215000, lastModified=2022-04-27 14:01:07.380484000, size=213, fileKey=(dev=801,ino=18837540)
14:21:08.091 DEBUG: file=null, size changed from -1 to 0 bytes
14:21:08.091 DEBUG: loading config FileBasedConfig[/root/.config/jgit/config]
14:21:08.092 DEBUG: readpipe [/usr/bin/git, --version],/usr/bin
14:21:08.100 DEBUG: readpipe may return 'git version 2.25.1'
14:21:08.100 DEBUG: remaining output:

14:21:08.102 DEBUG: readpipe [/usr/bin/git, config, --system, --edit],/usr/bin
14:21:08.109 DEBUG: readpipe may return '/etc/gitconfig'
14:21:08.109 DEBUG: remaining output:

14:21:08.111 DEBUG: file=/root/.config/jgit/config, isRacyClean=false, read=2022-04-27 14:21:08.091934000, lastModified=1970-01-01 00:00:00.000000000, delta=1651069268091934000 ns, racy<=2500000000 ns
14:21:08.111 DEBUG: file=/root/.config/jgit/config, is unmodified
14:21:08.111 DEBUG: file=null, size changed from -1 to 126 bytes
14:21:08.111 DEBUG: loading config FileBasedConfig[/etc/gitconfig]
14:21:08.112 DEBUG: file=/etc/gitconfig, create new FileSnapshot: lastRead=2022-04-27 14:21:08.111888000, lastModified=2022-04-21 13:42:25.000000000, size=126, fileKey=(dev=38,ino=91756)
14:21:08.112 DEBUG: file=/root/.config/jgit/config, isRacyClean=false, read=2022-04-27 14:21:08.091934000, lastModified=1970-01-01 00:00:00.000000000, delta=1651069268091934000 ns, racy<=2500000000 ns
14:21:08.112 DEBUG: file=/root/.config/jgit/config, is unmodified
14:21:08.112 DEBUG: file=/etc/gitconfig, isRacyClean=false, read=2022-04-27 14:21:08.111888000, lastModified=2022-04-21 13:42:25.000000000, delta=520723111888000 ns, racy<=2500000000 ns
.....
14:21:08.234 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, is racily clean
14:21:08.234 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, create new FileSnapshot: lastRead=2022-04-27 14:21:08.234731000, lastModified=2022-04-27 14:21:08.229687000, size=1, fileKey=(dev=801,ino=18841639)
14:21:08.234 DEBUG: file=/root/.config/jgit/config, isRacyClean=false, read=2022-04-27 14:21:08.091934000, lastModified=1970-01-01 00:00:00.000000000, delta=1651069268091934000 ns, racy<=2500000000 ns
14:21:08.234 DEBUG: file=/root/.config/jgit/config, is unmodified
14:21:08.234 DEBUG: file=/etc/gitconfig, isRacyClean=false, read=2022-04-27 14:21:08.111888000, lastModified=2022-04-21 13:42:25.000000000, delta=520723111888000 ns, racy<=2500000000 ns
14:21:08.234 DEBUG: file=/etc/gitconfig, is unmodified
14:21:08.234 DEBUG: file=/root/.gitconfig, isRacyClean=false, read=2022-04-27 14:21:08.113033000, lastModified=2022-04-21 13:42:25.000000000, delta=520723113033000 ns, racy<=2500000000 ns
14:21:08.235 DEBUG: file=/root/.gitconfig, is unmodified
14:21:08.235 DEBUG: file=/bitrise/src/.git/config, isRacyClean=false, read=2022-04-27 14:21:08.113614000, lastModified=2022-04-27 14:01:07.380484000, delta=1200733130000 ns, racy<=2500000000 ns
14:21:08.235 DEBUG: file=/bitrise/src/.git/config, is unmodified
14:21:08.235 DEBUG: Thread[main,5,main]: cannot measure timestamp resolution of unborn directory /bitrise/src/.git/refs/remotes/upstream
14:21:08.235 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, isRacyClean=true, read=2022-04-27 14:21:08.234731000, lastModified=2022-04-27 14:21:08.229687000, delta=5044000 ns, racy<=2500000000 ns
14:21:08.235 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, is racily clean
14:21:08.235 WARN: Could not find ref: master in refs/heads, refs/remotes/upstream or refs/remotes/origin
14:21:08.235 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, create new FileSnapshot: lastRead=2022-04-27 14:21:08.235671000, lastModified=2022-04-27 14:21:08.229687000, size=1, fileKey=(dev=801,ino=18841639)
14:21:08.236 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, isRacyClean=true, read=2022-04-27 14:21:08.235671000, lastModified=2022-04-27 14:21:08.229687000, delta=5984000 ns, racy<=2500000000 ns
14:21:08.236 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, is racily clean
14:21:08.236 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, create new FileSnapshot: lastRead=2022-04-27 14:21:08.236873000, lastModified=2022-04-27 14:21:08.229687000, size=1, fileKey=(dev=801,ino=18841639)
14:21:08.237 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, lastModified changed from 2022-04-27 14:21:08.229687000 to 2022-04-27 14:21:08.233688000
14:21:08.238 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, create new FileSnapshot: lastRead=2022-04-27 14:21:08.238255000, lastModified=2022-04-27 14:21:08.233688000, size=1, fileKey=(dev=801,ino=18841639)
14:21:08.238 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, isRacyClean=true, read=2022-04-27 14:21:08.238255000, lastModified=2022-04-27 14:21:08.233688000, delta=4567000 ns, racy<=2500000000 ns
14:21:08.239 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, is racily clean
14:21:08.239 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, create new FileSnapshot: lastRead=2022-04-27 14:21:08.239476000, lastModified=2022-04-27 14:21:08.233688000, size=1, fileKey=(dev=801,ino=18841639)
14:21:08.240 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, isRacyClean=true, read=2022-04-27 14:21:08.239476000, lastModified=2022-04-27 14:21:08.233688000, delta=5788000 ns, racy<=2500000000 ns
14:21:08.240 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, is racily clean
14:21:08.240 DEBUG: file=/bitrise/src/.git/.probe-08dbcf2c-b101-49df-8d73-63ce5349628b, create new FileSnapshot: lastRead=2022-04-27 14:21:08.240622000, lastModified=2022-04-27 14:21:08.233688000, size=1, fileKey=(dev=801,ino=18841639)
14:21:08.240 INFO: SCM collecting changed files in the branch (done) | time=166ms
14:21:08.240 DEBUG: SCM information about changed files in the branch is not available

[ganncamp (G Ann Campbell)]

Actually, these are interesting:

@vicsonvictor

14:21:08.235 WARN: Could not find ref: master in refs/heads, refs/remotes/upstream or refs/remotes/origin
...
14:21:08.240 DEBUG: SCM information about changed files in the branch is not available`

With the last one probably most relevant. If you look hard enough in the flood of information, here it is telling you that it can’t find the relevant information to know what’s “new” in the analysis.

 
Ann

[vicsonvictor]

Yep. I can infer that Is there anything else in the SONAR code/logs(attached files) which offers any clues to why we are seeing that line? Or how to not run into that case? Thanks.

[ganncamp (G Ann Campbell)]

Hi,

I think the best thing is to start from a vanilla checkout of the PR’s branch.

 
Ann