SonarScanner - Scan runs with ignore only files

Dennis_DECA · October 11, 2023, 2:35pm

Must-share information (formatted with Markdown):

SonarQube server 9.9.1.69595 LTS
SonarScanner 5.0.1.3006

Setup:
Github.com pull request scan. Self hosted Github runner.
Workflow snippet:

      - name: SonarQube Scan
        if: env.RUN_SonarQube == 'true'
        uses: sonarsource/sonarqube-scan-action@master
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
          SONAR_HOST_URL: ${{ vars.SONAR_HOST }}
          SONAR_SCANNER_OPTS: -Xmx6500m
        with:
          projectBaseDir: core
          args: >
            -Dsonar.projectBaseDir=/github/workspace/core
            -Dsonar.working.directory=/github/workspace/core/.scannerwork
            -Dsonar.java.skipUnchanged=true

Problem:
The GitHub PR scan only contains files that are excluded/ignored. Yet the scan is still running? Why is this? Can this be fixed?
Only file in PR: AAAARoot_bbbb/documents/datasource/CcccCccc.xml
This should be covered by:

**/documents/**/*.*

Log:

 with:
    projectBaseDir: core
    args: -Dsonar.projectBaseDir=/github/workspace/core -Dsonar.working.directory=/github/workspace/core/.scannerwork -Dsonar.java.skipUnchanged=true
  
  env:
    RUN_SonarQube: true
    SKIP_SonarQube: false
    SONAR_TOKEN: ***
    SONAR_HOST_URL: http://sonar.AAAAA.be:9000
    SONAR_SCANNER_OPTS: -Xmx6500m
/usr/bin/docker run --name e99f529aa0f3ed6a404b56ac7ae8ff9b8d8bf2_847ef2 --label e99f52 --workdir /github/workspace --rm -e "RUN_SonarQube" -e "SKIP_SonarQube" -e "SONAR_TOKEN" -e "SONAR_HOST_URL" -e "SONAR_SCANNER_OPTS" -e "INPUT_PROJECTBASEDIR" -e "INPUT_ARGS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e GITHUB_ACTIONS=true -e CI=true --entrypoint "/entrypoint.sh" -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/opt/github-runner/action-runner/_work/_temp/_github_home":"/github/home" -v "/opt/github-runner/action-runner/_work/_temp/_github_workflow":"/github/workflow" -v "/opt/github-runner/action-runner/_work/_temp/_runner_file_commands":"/github/file_commands" -v "/opt/github-runner/action-runner/_work/CCCC-core/CCCC-core":"/github/workspace" e99f52:9aa0f3ed6a404b56ac7ae8ff9b8d8bf2 -Dsonar.projectBaseDir=/github/workspace/core -Dsonar.working.directory=/github/workspace/core/.scannerwork -Dsonar.java.skipUnchanged=true
WARN: Property 'sonar.projectBaseDir' with value 'core' is overridden with value '/github/workspace/core'
INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /github/workspace/core/sonar-project.properties
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.8 Alpine (64-bit)
INFO: Linux 5.15.0-69-generic amd64
INFO: SONAR_SCANNER_OPTS=-Xmx6500m
INFO: User cache: /opt/sonar-scanner/.sonar/cache
INFO: Analyzing on SonarQube server 9.9.1.69595
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Load global settings
INFO: Load global settings (done) | time=185ms
INFO: Server id: 3ECB473A-AWowj3V-GjdYNLPaObtA
INFO: User cache: /opt/sonar-scanner/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=97ms
INFO: Load/download plugins (done) | time=22791ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Process project properties (done) | time=18ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=2ms
INFO: Project key: CCCC:core:bamboo:develop
INFO: Base dir: /github/workspace/core
INFO: Working dir: /github/workspace/core/.scannerwork
INFO: Load project settings for component key: 'CCCC:core:bamboo:develop'
INFO: Load project settings for component key: 'CCCC:core:bamboo:develop' (done) | time=107ms
INFO: Load project branches
INFO: Load project branches (done) | time=116ms
INFO: Load branch configuration
INFO: Detected branch/PR in 'GitHub Action'
INFO: Auto-configuring pull request '19125'
INFO: Load branch configuration (done) | time=9ms
INFO: Auto-configuring with CI 'Github Actions'
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=138ms
INFO: Load active rules
INFO: Load active rules (done) | time=4302ms
INFO: Load analysis cache
INFO: Load analysis cache | time=2934ms
INFO: Pull request 19125 for merge into develop from DDDD/ABC-123-Added-CCCCC
INFO: Load project repositories
INFO: Load project repositories (done) | time=709ms
INFO: SCM collecting changed files in the branch
INFO: Merge base sha1: AAAAA
INFO: SCM collecting changed files in the branch (done) | time=1396ms
INFO: Indexing files...
INFO: Project configuration:
INFO:  Excluded sources: **/*Stub.java, **/*.jar, **/*Test.java, _java/source/be/AAAAA/**/general/*Exceptions.java, **/documents/**/*.*, CCCCRoot*/query/**/*.*
INFO: 17888 files indexed
INFO: 2908 files ignored because of inclusion/exclusion patterns
INFO: 1 file ignored because of scm ignore settings
INFO: Quality profile for java: AAAAA Web
INFO: Quality profile for json: Sonar way
INFO: Quality profile for xml: Sonar way
INFO: Quality profile for yaml: Sonar way
INFO: ------------- Run sensors on module CCCC Core Bamboo develop
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=105ms
INFO: Sensor JavaSensor [java]
INFO: Configured Java source version (sonar.java.source): 8
INFO: JavaClasspath initialization
INFO: JavaClasspath initialization (done) | time=4ms
INFO: JavaTestClasspath initialization
INFO: JavaTestClasspath initialization (done) | time=2ms
INFO: The Java analyzer is running in a context where unchanged files can be skipped. Full analysis is performed for changed files, optimized analysis for unchanged files.
INFO: Server-side caching is enabled. The Java analyzer was able to leverage cached data from previous analyses for 13462 out of 17736 files. These files will not be parsed.
INFO: Using ECJ batch to parse 4274 Main java source files with batch size 340 KB.
INFO: Starting batch processing.
INFO: 0% analyzed
INFO: 6% analyzed

ganncamp · October 12, 2023, 5:14pm

Hi,

It’s not clear to me what the problem is. You invoked analysis and it ran.

Did you expect the scanner, as a whole, to shut down? It doesn’t work that way. The scanner invokes each analyzer, which then has the opportunity to either run or say “Nothing for me to do here. I’m quitting.” And then it’s the next one’s turn to do the same thing.

HTH,
Ann

Dennis_DECA · October 13, 2023, 11:57am

Hi Ann,

Ok, thank you for clarifying. In this case I believe it analyzers are processing more than expected, because it finds a bigger diff was detected:

INFO: Server-side caching is enabled. The Java analyzer was able to leverage cached data from previous analyses for 13462 out of 17736 files. These files will not be parsed.

My expectation was:
Only Ignored files detected, cancel any analyzers, send empty/positive result to SQ server.

Reality:
Changes were detected, X files should be ignored. Provide this information to the analyzers who have logic to ignore/skip files during the analysis. Provide result to SQ server.

ganncamp · October 13, 2023, 1:22pm

Hi,

Sorry, but I really don’t understand.

I guess you’re questioning why changing only a few files invalidated cache for ~4k files?

If so… I’m not conversant on the details, but I believe this has to do with the network of interdependencies among your files.

HTH,
Ann

system · October 20, 2023, 1:23pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.