Master - Overall code 'loses' issues and doesn't reflect full repo scan result

  • ALM used: Github
  • CI system used: Github
  • Scanner command used when applicable:
    PR request scan(works as expected):
- name: "Push reports to Sonarcloud"
  run: >
   mvn ${MAVEN_CLI_OPTS} -B sonar:sonar -pl "${EXCLUDED_MODULES_LIST}"
          -Dsonar.token=${SONAR_TOKEN}
          -Dsonar.pullrequest.base=master
          -Dsonar.pullrequest.key=${{ github.event.pull_request.number }}
          -Dsonar.pullrequest.branch=${{ github.event.pull_request.head.ref }}
          -Dsonar.organization=MASKED
          -Dsonar.projectKey=MASKED
          -Dsonar.host.url=https://sonarcloud.io/ 
          -Dsonar.branch.name=${{env.GITHUB_HEAD_REF}}
          -Dsonar.kotlin.detekt.reportPaths=target/detekt.xml
          -Dsonar.dependencyCheck.reportPath=target/dependency-check-report.xml
          -Dsonar.kotlin.source.version="${KOTLIN_VERSION}"
          -Dsonar.kotlin.threads="${KOTLIN_SONAR_THREADS}"
          -Dsonar.kotlin.skipUnchanged=true

Merge to master scan:

      - name: "Push reports to SonarCloud (full scan)"
        run: >
          mvn ${MAVEN_CLI_OPTS} -B verify -Pcoverage sonar:sonar -pl "${EXCLUDED_MODULES_LIST}"
          -DskipSpotbugs=false
          -Dsonar.login=${SONAR_TOKEN}
          -Dsonar.organization=MASKED 
          -Dsonar.projectKey=MASKED
          -Dsonar.host.url=https://sonarcloud.io/ 
          -Dsonar.branch.name=master
          -Dsonar.kotlin.detekt.reportPaths=target/detekt.xml
          -Dsonar.dependencyCheck.reportPath=target/dependency-check-report.xml
          -Dsonar.kotlin.source.version="${KOTLIN_VERSION}"
          -Dsonar.kotlin.threads="${KOTLIN_SONAR_THREADS}"
          -Dsonar.kotlin.skipUnchanged=true
  • Languages of the repository: kotlin, java

  • Only if the SonarCloud project is public - project is not public

  • Error observed:
    No clear errors, the problem that Master - Overall code doesn’t display all code smells issues, shows approximately 10-15%, for some reasons results are ‘lost’ until next time workaround applied.

  • Steps to reproduce - for my project is just normal run, but it’s not public.Not sure 100% - next merge to master after workaround or some time after workaround, but most probably next merge, as I see issues dropped after 1 line commit change even.

  • Potential workaround:
    Workaround PR scan executed to get all repo results. A bit changed PR Sonar scan is executed for PR with specific branch.
    Without workaround - Master overall code shows 15% of all code issues, after it - all data is present until next normal merge to master.

 - name: "Push reports to Sonarcloud"
        run: >
          mvn ${MAVEN_CLI_OPTS} -B clean verify sonar:sonar -pl "${EXCLUDED_MODULES_LIST}"
          -Dsonar.login=${SONAR_TOKEN}
          -Dsonar.organization=Masked
          -Dsonar.projectKey=Masked
          -Dsonar.host.url=https://sonarcloud.io/
          -Dsonar.branch.name=master
          -Dsonar.kotlin.detekt.reportPaths=target/detekt.xml
          -Dsonar.java.spotbugs.reportPaths=target/spotbugsXml.xml
          -Dsonar.kotlin.source.version="${KOTLIN_VERSION}"
          -Dsonar.kotlin.threads="${KOTLIN_SONAR_THREADS}"

Hi,

Are you saying that 10-15% of issues are lost, or that only 10-15% of issues are retained?

Can you characterize the lost issues? Are they from a particular rule? A particular type of rule?

It seems like the lost issues come back without intervention in the next analysis?

Can you share an analysis log from a ‘lost issues’ analysis, as well as one from a ‘returned issues’ analysis?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

 
Thx,
Ann

Hi, only 10-15% retained.
Looks like it’s loosing issues detected before, and last PR results are still displayed in Overall Code.
Can you guide me how to share logs not in public forum?
What email can I use for example?

Hi,

Feel free to redact your logs as necessary.

Also, can you clarify these points, please?

It would also help if you could compare the SonarScanner Context from an analysis with all the issues and one that’s missing the majority to see what, if anything, changes between them. You’ll find it under Administration → Background Tasks → [row dots menu]

 
Ann

I haven’t identified any patterns for issues, the only pattern I see:
First scan after “Workaround” mentioned above - brings all results back, but once next PR from regular run got merged and ‘normal’ sonar scan executed - issues got lost from Overall code section.

Will add SonarScanner Context and scanner log later today.

Sorry for the delay.
I had few regular runs and all results still in place but after I check activity section, I see results are remaining for some time. Not lost immediately after regular run.

Comparing SonarScannerContext for regular run with lost issues and without - I haven’t found difference.
But there is difference between regular run and workaround:

Regular run(same for the one with lost issues):

Project scanner properties(diff not present in workaround execution):
 
  - sonar.coverage.jacoco.xmlReportPaths=${SONAR_XML_REPORT_PATHS}
  - sonar.kotlin.skipUnchanged=true
  - sonar.kotlin.source.version=1.7
  - sonar.kotlin.threads=2
  - sonar.scanner.appVersion=3.11.0.3922/3.8.8

Workaround run:

Project scanner properties:
  - sonar.java.spotbugs.reportPaths=target/spotbugsXml.xml
  - sonar.scanner.appVersion=3.10.0.2594/3.8.8

Can it be
- sonar.kotlin.skipUnchanged=true causing the issue after some time?

Scanner log for regular run with all issues detected(single module):

2024-03-13T13:23:58.6790958Z 13:23:58.678 [INFO] 13:23:58.678 ------------- Run sensors on module [Redacted]

2024-03-13T13:23:58.8787284Z 13:23:58.878 [INFO] 13:23:58.878 Sensor ThymeLeaf template sensor [securityjavafrontend]

2024-03-13T13:23:58.8789144Z 13:23:58.878 [INFO] 13:23:58.878 Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms

2024-03-13T13:23:58.8791917Z 13:23:58.879 [INFO] 13:23:58.879 Sensor JaCoCo XML Report Importer [jacoco]

2024-03-13T13:23:58.8796382Z 13:23:58.879 [WARNING] 13:23:58.879 No coverage report can be found with sonar.coverage.jacoco.xmlReportPaths='${SONAR_XML_REPORT_PATHS}'.

Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml

2024-03-13T13:23:58.8799459Z 13:23:58.879 [INFO] 13:23:58.879 Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.

2024-03-13T13:23:58.9191989Z 13:23:58.918 [INFO] 13:23:58.918 Sensor JaCoCo XML Report Importer [jacoco] (done) | time=39ms

2024-03-13T13:23:58.9193673Z 13:23:58.919 [INFO] 13:23:58.919 Sensor Kotlin Sensor [kotlin]

2024-03-13T13:23:58.9236150Z 13:23:58.923 [INFO] 13:23:58.923 Only analyzing 17 changed Kotlin files out of 17.

2024-03-13T13:23:58.9354526Z 13:23:58.935 [INFO] 13:23:58.935 17 source files to be analyzed

2024-03-13T13:24:05.8400662Z 13:24:05.839 [INFO] 13:24:05.839 17/17 source files have been analyzed

2024-03-13T13:24:05.8406338Z 13:24:05.840 [INFO] 13:24:05.840 Sensor Kotlin Sensor [kotlin] (done) | time=6921ms

2024-03-13T13:24:05.8412676Z 13:24:05.841 [INFO] 13:24:05.841 Sensor Gradle Sensor [kotlin]

2024-03-13T13:24:05.8419606Z 13:24:05.841 [INFO] 13:24:05.841 Sensor Gradle Sensor [kotlin] (done) | time=0ms

2024-03-13T13:24:05.8425348Z 13:24:05.842 [INFO] 13:24:05.842 Sensor KotlinSurefireSensor [kotlin]

2024-03-13T13:24:05.8432872Z 13:24:05.843 [INFO] 13:24:05.843 parsing [/home/runner/work/[Redacted][Redacted]/target/surefire-reports]

2024-03-13T13:24:06.0274699Z 13:24:06.027 [INFO] 13:24:06.027 Searching for com/[Redacted]

2024-03-13T13:24:06.0411099Z 13:24:06.040 [INFO] 13:24:06.040 Sensor KotlinSurefireSensor [kotlin] (done) | time=198ms

2024-03-13T13:24:06.0415349Z 13:24:06.041 [INFO] 13:24:06.041 Sensor Import of detekt issues [kotlin]

2024-03-13T13:24:06.0420458Z 13:24:06.041 [INFO] 13:24:06.041 Importing /home/runner/work/[Redacted][Redacted]/target/detekt.xml

2024-03-13T13:24:06.0435279Z 13:24:06.043 [WARNING] 13:24:06.043 No input file found for /home/runner/work/[REDACTED] No detekt issues will be imported on this file.

2024-03-13T13:24:06.0574399Z 13:24:06.057 [INFO] 13:24:06.057 Sensor Import of detekt issues [kotlin] (done) | time=16ms

2024-03-13T13:24:06.0577995Z 13:24:06.057 [INFO] 13:24:06.057 Sensor TextAndSecretsSensor [text]

2024-03-13T13:24:06.1990014Z 13:24:06.198 [INFO] 13:24:06.198 34 source files to be analyzed

2024-03-13T13:24:06.7208628Z 13:24:06.720 [INFO] 13:24:06.720 34/34 source files have been analyzed

2024-03-13T13:24:06.7210679Z 13:24:06.720 [INFO] 13:24:06.720 Sensor TextAndSecretsSensor [text] (done) | time=663ms

2024-03-13T13:24:06.7212533Z 13:24:06.720 [INFO] 13:24:06.720 Sensor IaC Docker Sensor [iac]

2024-03-13T13:24:06.7472012Z 13:24:06.743 [INFO] 13:24:06.743 0 source files to be analyzed

2024-03-13T13:24:06.7473706Z 13:24:06.743 [INFO] 13:24:06.743 0/0 source files have been analyzed

2024-03-13T13:24:06.7475518Z 13:24:06.743 [INFO] 13:24:06.743 Sensor IaC Docker Sensor [iac] (done) | time=23ms

2024-03-13T13:24:06.7477167Z 13:24:06.743 [INFO] 13:24:06.743 Sensor Serverless configuration file sensor [security]

2024-03-13T13:24:06.7479436Z 13:24:06.743 [INFO] 13:24:06.743 0 Serverless function entries were found in the project

2024-03-13T13:24:06.7481109Z 13:24:06.743 [INFO] 13:24:06.743 0 Serverless function handlers were kept as entrypoints

2024-03-13T13:24:06.7483167Z 13:24:06.743 [INFO] 13:24:06.743 Sensor Serverless configuration file sensor [security] (done) | time=0ms

2024-03-13T13:24:06.7484840Z 13:24:06.743 [INFO] 13:24:06.743 Sensor AWS SAM template file sensor [security]

2024-03-13T13:24:06.7486731Z 13:24:06.744 [INFO] 13:24:06.744 Sensor AWS SAM template file sensor [security] (done) | time=1ms

2024-03-13T13:24:06.7488407Z 13:24:06.744 [INFO] 13:24:06.744 Sensor AWS SAM Inline template file sensor [security]

2024-03-13T13:24:06.7490417Z 13:24:06.744 [INFO] 13:24:06.744 Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms

2024-03-13T13:25:35.6673370Z 13:25:35.656 [INFO] 13:25:35.656 ------------- Run sensors on project

2024-03-13T13:25:35.6674474Z 13:25:35.659 [INFO] 13:25:35.659 Sensor Zero Coverage Sensor

2024-03-13T13:25:35.6829072Z 13:25:35.682 [INFO] 13:25:35.682 Sensor Zero Coverage Sensor (done) | time=23ms

2024-03-13T13:25:35.6832917Z 13:25:35.683 [INFO] 13:25:35.683 Sensor Java CPD Block Indexer

2024-03-13T13:25:35.7721063Z 13:25:35.771 [INFO] 13:25:35.771 Sensor Java CPD Block Indexer (done) | time=88ms

2024-03-13T13:25:35.8311285Z 13:25:35.830 [INFO] 13:25:35.830 SCM Publisher SCM provider for this project is: git

2024-03-13T13:25:35.8320867Z 13:25:35.831 [INFO] 13:25:35.831 SCM Publisher 11 source files to be analyzed

2024-03-13T13:25:37.0025176Z 13:25:37.002 [INFO] 13:25:37.002 SCM Publisher 11/11 source files have been analyzed (done) | time=1170ms

2024-03-13T13:25:37.3384740Z 13:25:37.338 [INFO] 13:25:37.338 CPD Executor 66 files had no CPD blocks

2024-03-13T13:25:37.3386479Z 13:25:37.338 [INFO] 13:25:37.338 CPD Executor Calculating CPD for 381 files

2024-03-13T13:25:37.4923600Z 13:25:37.492 [INFO] 13:25:37.492 CPD Executor CPD calculation finished (done) | time=153ms

2024-03-13T13:25:39.1495015Z 13:25:39.149 [INFO] 13:25:39.149 Analysis report generated in 1611ms, dir size=6 MB

2024-03-13T13:25:40.4202592Z 13:25:40.420 [INFO] 13:25:40.420 Analysis report compressed in 1270ms, zip size=2 MB

2024-03-13T13:25:42.1870001Z 13:25:42.186 [INFO] 13:25:42.186 Analysis report uploaded in 1766ms

2024-03-13T13:25:42.1886213Z 13:25:42.188 [INFO] 13:25:42.188 ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=[Redacted]&branch=master

2024-03-13T13:25:42.1890837Z 13:25:42.188 [INFO] 13:25:42.188 Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report

2024-03-13T13:25:42.1895233Z 13:25:42.189 [INFO] 13:25:42.189 More about the report processing at https://sonarcloud.io/api/ce/task?id=AY43_XfMNDGIViqnzkcu

2024-03-13T13:25:44.0977731Z 13:25:44.097 [INFO] 13:25:44.097 Sensor cache published successfully

2024-03-13T13:25:44.2274876Z 13:25:44.227 [INFO] 13:25:44.227 Analysis total time: 2:56.227 s

2024-03-13T13:25:44.2280305Z 13:25:44.227 [INFO] ------------------------------------------------------------------------

2024-03-13T13:25:44.2281898Z 13:25:44.227 [INFO] Reactor Summary for [Redacted]

2024-03-13T13:25:44.2473986Z 13:25:44.229 [INFO] ------------------------------------------------------------------------

2024-03-13T13:25:44.2474951Z 13:25:44.229 [INFO] BUILD SUCCESS

Is not much different from the same regular run with lost issues(same module):

2024-03-11T13:39:23.1274880Z 13:39:23.127 [INFO] ------------- Run sensors on module [Redacted]

2024-03-11T13:39:23.1622566Z 13:39:23.161 [INFO] Sensor ThymeLeaf template sensor [securityjavafrontend]

2024-03-11T13:39:23.1626518Z 13:39:23.162 [INFO] Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms

2024-03-11T13:39:23.1630460Z 13:39:23.162 [INFO] Sensor JaCoCo XML Report Importer [jacoco]

2024-03-11T13:39:23.1636854Z 13:39:23.163 [WARNING] No coverage report can be found with sonar.coverage.jacoco.xmlReportPaths='${SONAR_XML_REPORT_PATHS}'.

Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml

2024-03-11T13:39:23.1643972Z 13:39:23.164 [INFO] Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.

2024-03-11T13:39:23.1824003Z 13:39:23.182 [INFO] Sensor JaCoCo XML Report Importer [jacoco] (done) | time=20ms

2024-03-11T13:39:23.1827845Z 13:39:23.182 [INFO] Sensor Kotlin Sensor [kotlin]

2024-03-11T13:39:23.2026391Z 13:39:23.202 [INFO] Only analyzing 0 changed Kotlin files out of 17.

2024-03-11T13:39:23.2031123Z 13:39:23.202 [INFO] Sensor Kotlin Sensor [kotlin] (done) | time=20ms

2024-03-11T13:39:23.2036623Z 13:39:23.203 [INFO] Sensor Gradle Sensor [kotlin]

2024-03-11T13:39:23.2041907Z 13:39:23.204 [INFO] Sensor Gradle Sensor [kotlin] (done) | time=1ms

2024-03-11T13:39:23.2046868Z 13:39:23.204 [INFO] Sensor KotlinSurefireSensor [kotlin]

2024-03-11T13:39:23.2052559Z 13:39:23.205 [INFO] parsing [/home/runner/work/[Redacted][Redacted]/target/surefire-reports]

2024-03-11T13:39:23.2332817Z 13:39:23.233 [INFO] Searching for com/[Redacted]

2024-03-11T13:39:23.2383856Z 13:39:23.238 [INFO] Sensor KotlinSurefireSensor [kotlin] (done) | time=34ms

2024-03-11T13:39:23.2387639Z 13:39:23.238 [INFO] Sensor Import of detekt issues [kotlin]

2024-03-11T13:39:23.2389367Z 13:39:23.238 [INFO] Importing /home/runner/work/[Redacted][Redacted]/target/detekt.xml

2024-03-11T13:39:23.2424303Z 13:39:23.242 [WARNING] No input file found for /home/runner/work/[Redacted] No detekt issues will be imported on this file.

024-03-11T13:39:23.2647062Z 13:39:23.255 [INFO] Sensor Import of detekt issues [kotlin] (done) | time=17ms

2024-03-11T13:39:23.2648208Z 13:39:23.255 [INFO] Sensor TextAndSecretsSensor [text]

2024-03-11T13:39:23.2858710Z 13:39:23.285 [INFO] 34 source files to be analyzed

2024-03-11T13:39:23.7633900Z 13:39:23.763 [INFO] 34/34 source files have been analyzed

2024-03-11T13:39:23.7635219Z 13:39:23.763 [INFO] Sensor TextAndSecretsSensor [text] (done) | time=508ms

2024-03-11T13:39:23.7636408Z 13:39:23.763 [INFO] Sensor IaC Docker Sensor [iac]

2024-03-11T13:39:23.7682102Z 13:39:23.768 [INFO] 0 source files to be analyzed

2024-03-11T13:39:23.7683567Z 13:39:23.768 [INFO] 0/0 source files have been analyzed

2024-03-11T13:39:23.7684990Z 13:39:23.768 [INFO] Sensor IaC Docker Sensor [iac] (done) | time=5ms

2024-03-11T13:39:23.7686588Z 13:39:23.768 [INFO] Sensor Serverless configuration file sensor [security]

2024-03-11T13:39:23.7688294Z 13:39:23.768 [INFO] 0 Serverless function entries were found in the project

2024-03-11T13:39:23.7689928Z 13:39:23.768 [INFO] 0 Serverless function handlers were kept as entrypoints

2024-03-11T13:39:23.7691788Z 13:39:23.768 [INFO] Sensor Serverless configuration file sensor [security] (done) | time=0ms

2024-03-11T13:39:23.7693496Z 13:39:23.768 [INFO] Sensor AWS SAM template file sensor [security]

2024-03-11T13:39:23.7695136Z 13:39:23.768 [INFO] Sensor AWS SAM template file sensor [security] (done) | time=0ms

2024-03-11T13:39:23.7696914Z 13:39:23.768 [INFO] Sensor AWS SAM Inline template file sensor [security]

2024-03-11T13:39:23.7698696Z 13:39:23.768 [INFO] Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms

2024-03-11T13:39:30.7043528Z 13:39:30.693 [INFO] ------------- Run sensors on project

2024-03-11T13:39:30.7044355Z 13:39:30.696 [INFO] Sensor Zero Coverage Sensor

2024-03-11T13:39:30.7045297Z 13:39:30.698 [INFO] Sensor Zero Coverage Sensor (done) | time=2ms

2024-03-11T13:39:30.7046217Z 13:39:30.698 [INFO] Sensor Java CPD Block Indexer

2024-03-11T13:39:30.7301872Z 13:39:30.730 [INFO] Sensor Java CPD Block Indexer (done) | time=31ms

2024-03-11T13:39:30.7632496Z 13:39:30.763 [INFO] SCM Publisher SCM provider for this project is: git

2024-03-11T13:39:30.7643262Z 13:39:30.764 [INFO] SCM Publisher 7 source files to be analyzed

2024-03-11T13:39:31.9373049Z 13:39:31.937 [INFO] SCM Publisher 7/7 source files have been analyzed (done) | time=1172ms

2024-03-11T13:39:32.0002193Z 13:39:31.999 [INFO] CPD Executor 66 files had no CPD blocks

2024-03-11T13:39:32.0007041Z 13:39:32.000 [INFO] CPD Executor Calculating CPD for 379 files

2024-03-11T13:39:32.1600412Z 13:39:32.159 [INFO] CPD Executor CPD calculation finished (done) | time=158ms

2024-03-11T13:39:33.6981854Z 13:39:33.697 [INFO] Analysis report generated in 1510ms, dir size=5 MB

2024-03-11T13:39:34.5585574Z 13:39:34.558 [INFO] Analysis report compressed in 860ms, zip size=1 MB

2024-03-11T13:39:35.6392220Z 13:39:35.638 [INFO] Analysis report uploaded in 1080ms

2024-03-11T13:39:35.6416874Z 13:39:35.641 [INFO] ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=[Redacted]&branch=master

2024-03-11T13:39:35.6421079Z 13:39:35.641 [INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report

2024-03-11T13:39:35.6425251Z 13:39:35.641 [INFO] More about the report processing at https://sonarcloud.io/api/ce/task?id=AY4tvXdTGtGanBvG-ynx

2024-03-11T13:39:38.8060782Z 13:39:38.805 [INFO] Sensor cache published successfully

2024-03-11T13:39:38.8827568Z 13:39:38.882 [INFO] Analysis total time: 1:06.698 s

The only difference - after the lost one - workaround was applied.

And missing issues also reported as “New Issues” in new code after workaround, so it looks Sonar forgets about old issues(they were present week ago, then lost and now under New Code -> New Issues):

New code definitions - “last 90 days”

Hi,

We’ve recently done an in-depth investigation of “flickering” issues. What we found was that in every case we looked at it traced back to analysis running with different settings/parameters. You should be able to confirm this by looking at the scanner context, which you’ll find in the project’s Background Tasks, in the row menu.

 
HTH,
Ann

Hi,
sorry, can you please clarify:

  • what exactly should I check in scanner context on top of mentioned difference between regular and workaround runs?
  • what should be changed to stop loosing results in future and get them in “new issues” after workaround run?

Hi,

Compare the values in the two different property sets to see what changes.

Make sure they run with exactly the same properties.

 
HTH,
Ann

On Friday again after last successful run at 12:38 master ‘lost’ results at 14:38.
Compared scanner context -
successful full result duration - 7.253s
duration for next master run with ‘lost’ results - 6.327s
Totally identical Sonar contexts, (compared with file diff in visual Studio)

Hi,

Could we have the logs from those two runs, as well as the contexts, redacted as necessary?

 
Thx,
Ann

Hi,
Is there any alternative way to share logs except public forum?
it’s approximately 25k lines per each run for Sonar step.
What I can confirm - issue never occurs if ‘merge to master’ scan doesn’t have

-Dsonar.kotlin.skipUnchanged=true

Will it be enough to provide logs since step sonar-maven-plugin:3.11.0.3922:sonar ? It’s just 2k of log lines.