Disappearing issues on Android project using Sonar Scanner

[BerkayMutlu]

Hello,

We are trying to add Sonar to our project.

Server: Community Edition v10.7
Sonar Scanner: SonarScanner 5.0.1.3006
Sonar Gradle Plugin: 6.2.0.5505

Our project is a multi-module Gradle project of an Android app, 1000+ modules, 99% is Kotlin the rest is a mix of Java, Python, Javascript.

We run:

• nightly on our trunk branch
• on each feature merge request

The initial spike (if I remember correctly) was the Sonar Scanner, the it dropped to low issue count using the same Sonar Scanner config, then we tried Sonar Gradle Plugin which made it jump back up to 6.9k issues and then we run Sonar Scanner again which removed all the previously reported issues and dropped it back to 70.

We tried the Gradle Plugin but it takes 25 mins to complete which is just too long to add it to our merge request pipeline. So we are trying to use Sonar Scanner instead.

Sonar Scanner completes in 7-8 minutes however while the Gradle Plugin finds 6.9k issues the Sonar Scanner CLI tool finds about 70. I don’t know whether our config and setup prevents finding issues or whether Sonar server app compares our version with the previous version and then deletes the previously reported issues on its own.

Only 15 issues in our 1M+ lines of code:

Warnings seem unrelated (we didn’t run Lint and Detekt its normal they’re missing):

We do these in this order

1. gradle assembleDebug
2. gradle testDebugUnitTest reportTestCoverage
3. sonar-scanner "${SONAR_ARGS[@]}" -Dsonar.gitlab.commit_sha=$COMMIT_DIFFS -Dsonar.gitlab.ref_name=$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME -Dsonar.gitlab.project_id=$CI_PROJECT_ID -Dsonar.pullrequest.key=$CI_MERGE_REQUEST_IID -Dsonar.pullrequest.branch=$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME -Dsonar.pullrequest.base=$CI_MERGE_REQUEST_TARGET_BRANCH_NAME 2>&1

We tried a few configs, last one that is working is below:

sonar.projectKey=our-project-key
sonar.sources=.
sonar.inclusions=**/src/main/**/*.java,**/src/main/**/*.kt
sonar.exclusions=**/build/**
sonar.tests=.
sonar.test.inclusions=**/src/test/**/*.java,**/src/test/**/*.kt,**/src/androidTest/**/*.java,**/src/androidTest/**/*.kt
sonar.java.binaries=.
sonar.java.libraries=.

sonar.kotlin.skipUnchanged=true
sonar.java.skipUnchanged=true

sonar.java.coveragePlugin=jacoco
sonar.junit.reportPaths=**/build/test-results/testDebugUnitTest/*.xml
sonar.dynamicAnalysis=reuseReports

sonar.coverage.jacoco.xmlReportPaths=**/build/reports/jacoco/reportTestCoverage/reportTestCoverage.xml

sonar.androidLint.reportPaths=**/build/reports/lint-results-debug.xml
sonar.kotlin.detekt.reportPaths=**/build/reports/detekt/debug.xml

This one also succeeds but with a similar result:

sonar.projectKey=our-key-that-is-obfuscated
sonar.exclusions=**/build/**,**/*.java,**/*.java
sonar.java.coveragePlugin=jacoco
sonar.junit.reportPaths=**/build/test-results/testDebugUnitTest/*.xml
sonar.coverage.jacoco.xmlReportPaths=**/build/reports/jacoco/reportTestCoverage/reportTestCoverage.xml
sonar.androidLint.reportPaths=**/build/reports/lint-results-debug.xml
sonar.kotlin.detekt.reportPaths=**/build/reports/detekt/debug.xml

Tried enabling java.binaries and java.libraries

sonar.java.binaries=**/build/intermediates/javac/**/classes,**/build/tmp/kotlin-classes/**
sonar.java.libraries=**/libs/**/*.jar

But the Sonar Scanners halts the scan due to missing files even though I can see there are files locally in these directories after compilation:

ERROR: Error during SonarScanner execution
java.lang.IllegalStateException: No files nor directories matching '**/build/intermediates/javac/**/classes'
	at org.sonar.java.classpath.AbstractClasspath.getFilesFromProperty(AbstractClasspath.java:128)
	at org.sonar.java.classpath.ClasspathForMain.init(ClasspathForMain.java:56)

I’m trying to find answers to these questions:

• is our setup finding issues correctly? is there a way to verify this?
• why previously reported issues are disappearing? is there a way to prevent this? can the “new code” setting on Sonar web app causing this? We were using " Previous version" setting but now trying “reference branch” (with our trunk) to see if it helps.
• what should be the values of sonar.java.binaries and sonar.java.libraries? on forums I see may people use **/target/classes what is target in this case? is that for a single-module project or a Maven project? Totally confused about these two properties. What happens when we don’t set them up? It’s also doable to ignore Java files entirely but would that affect Kotlin too?
• Do we need to compile our project before running Sonar Scanner? If yes, which Gradle tasks do you recommend for a Kotlin, Java Android Gradle Project? Would gradle assembleDebug is enough or should we run other tasks too?

I know this is a lot of questions. But honestly I’m lost on Sonar’s behavior of disappearing issues. Sonar Scanner is also very eager to halt the scan, I would be more happy to see a warning and let it continue.

Sonar Scanner logs (without -X flag) doesn’t show a related warning or error log

Thank you for your time and help
Berkay.

Related posts:

• How to provide compiled Java classes with sonar.java.binaries property using the Sonar Gradle Plugin - #2 by ganncamp
• Missing Jar files when running Sonar Gradle Plugin with Java/Kotlin only modules - #4 by BerkayMutlu

[Colin]

Hey @BerkayMutlu

It looks like you’re trying to run PR analysis on the Community Edition of SonarQube. The use of plugins to enable this functionality in SonarQube’s Community Build is not supported in this Community. I’ll leave you with a few notes:

• If SonarQube no longer detects issues in a new analysis, those issues are closed. That appears to be the behavior you’re experiencing here.
• Kotlin analysis has improved in the latest versions. You should upgrade to a supported version (the only supported version of Community Build is the latest version, 25.5).
• The SonarScanner for Gradle sets a lot of important properties to make sure the analysis is correct. I really encourage you to use the SonarScanner for Gradle. That said, you can try and mimic how the SonarScanner for Gradle sets properties like sonar.java.libraries and sonar.java.binaries by browsing the SonarScanner Context of the background task of the analysis submitted by the SonarScanner for Gradle
• sonar.java.libraries and sonar.java.binaries expect compiled bytecode (of dependencies and your source code respectively), so you should compile your project before analysis.
• The initial analysis might be long, but you should make sure that the PR analysis is using the cache of the target branch. That said, I have no idea with the plugin you’re using to enable branch/PR analysis of analysis caching is supported.