Caching sonarscanner

oana.marina · July 25, 2024, 3:10pm

I don’t think caching the sonar scanner works.
We are using:

SonarQube Enterprise Edition Version 9.9 (build 65466)
‘dotnet-sonarscanner’ (version ‘6.2.0’)
code is C#

This is the summary of the workflow, it runs on a self-hosted server, always the same server:

   - name: Cache SonarQube packages
        uses: actions/cache@v4.0.2
        with:
          path: ~\.sonar\cache
          key: ${{ runner.os }}-sonar
          restore-keys: ${{ runner.os }}-sonar

      - name: Cache SonarQube scanner
        id: cache-sonar-scanner
        uses: actions/cache@v4.0.2
        with:
          path: .\.sonar\scanner
          key: ${{ runner.os }}-sonar-scanner
          restore-keys: ${{ runner.os }}-sonar-scanner

      - name: Install SonarQube scanner
        if: steps.cache-sonar-scanner.outputs.cache-hit != 'true'
        shell: powershell
        run: |
            New-Item -Path .\.sonar\scanner -ItemType Directory
            dotnet tool update dotnet-sonarscanner --tool-path .\.sonar\scanner
      - name: SonarQube Analysis Start
        run: |
          dotnet tool install --global dotnet-coverage
          .\.sonar\scanner\dotnet-sonarscanner begin /d:sonar.host.url="${{secrets.SONARQUBE_HOST}}"
   
    -build
   -run tests

     - name: SonarQube Analysis End
        if: success() || failure()
        run: .\.sonar\scanner\dotnet-sonarscanner end /d:sonar.login="${{secrets.SONARQUBE_TOKEN}}"

And this is from logs of the second run of the workflow and it looks like nothing is cached

Run actions/cache@v4.0.2
  with:
    path: ~\.sonar\cache
    key: Windows-sonar
    restore-keys: Windows-sonar
    enableCrossOsArchive: false
    fail-on-cache-miss: false
    lookup-only: false
    save-always: false
  env:
    ENVIRONMENT: Development
    buildConfiguration: Release
    VERSION_NUMBER: 1.0
    NUGET: C:\actions-runner\_work\_tool\nuget.exe\5.11.6\x64/nuget.exe
Cache not found for input keys: Windows-sonar, Windows-sonar

Run actions/cache@v4.0.2
  with:
    path: .\.sonar\scanner
    key: Windows-sonar-scanner
    restore-keys: Windows-sonar-scanner
    enableCrossOsArchive: false
    fail-on-cache-miss: false
    lookup-only: false
    save-always: false
  env:
    ENVIRONMENT: Development
    buildConfiguration: Release
    VERSION_NUMBER: 1.0
    NUGET: C:\actions-runner\_work\_tool\nuget.exe\5.11.6\x64/nuget.exe
Cache not found for input keys: Windows-sonar-scanner, Windows-sonar-scanner

ganncamp · July 26, 2024, 4:11pm

Hi,

Can you help me understand why you’re trying to download and cache the SonarScanner for .NET from your pipeline, rather than just installing it on your build agent?

Thx,
Ann

oana.marina · July 27, 2024, 8:39pm

Mainly because our IT department has a nasty habit of running security software that uninstalls things that seem suspicious. Running the cache and installing the sonarscanner is a total of 4s, so it’s not a big deal.

ganncamp · July 29, 2024, 11:39am

Hi,

Could this be where your cache is going?

Ann

oana.marina · July 29, 2024, 3:36pm

I don’t think so. If I run the workflow multiple times back to back it’s the same thing.

Basically I have 3 workflows, same actions done but for different code. One shows cache, the other two don’t. The one that shows the cache (intermittently) is run on the PR, and on a windows-latest runner. It doesn’t show it for all PRs, and since it’s a new runner every time I don’t understand where the cache is coming from.

Run actions/cache@v4.0.2
Received 0 of 283207775 (0.0%), 0.0 MBs/sec
Received 83886080 of 283207775 (29.6%), 40.0 MBs/sec
Received 201326592 of 283207775 (71.1%), 63.5 MBs/sec
Received 279013471 of 283207775 (98.5%), 66.0 MBs/sec
Received 279013471 of 283207775 (98.5%), 52.9 MBs/sec
Cache Size: ~270 MB (283207775 B)
"C:\Program Files\Git\usr\bin\tar.exe" -xf D:/a/_temp/e3b8b408-23dd-49a3-9852-9f623c34eb2f/cache.tzst -P -C D:/a/app/source --force-local --use-compress-program "zstd -d"
Received 283207775 of 283207775 (100.0%), 44.8 MBs/sec
Cache restored successfully
Cache restored from key: Windows-sonar
2s
Run actions/cache@v4.0.2
Received 0 of 2794594 (0.0%), 0.0 MBs/sec
Cache Size: ~3 MB (2794594 B)
"C:\Program Files\Git\usr\bin\tar.exe" -xf D:/a/_temp/332cb4eb-af83-4e54-a486-bb81067e3637/cache.tzst -P -C D:/a/app/source --force-local --use-compress-program "zstd -d"
Cache restored successfully
Cache restored from key: Windows-sonar-scanner

The other two workflows are run on main, scheduled once a day. One is on a windows-latest, the other on a self-hosted runner. None show the cache.

ganncamp · July 29, 2024, 5:35pm

Hi,

What DevOps platform are we dealing with here, and where is your cache supposed to live? Because it sounds like your build hops around on different build agents.

Ann

oana.marina · July 29, 2024, 7:52pm

We are using github.
Workflow1: runs on main, once a day, self-hosted runner
Workflow2: runs on main, once a day, github hosted runner, windows-latest
Workflow3: runs on PR, github hosted runner, windows-latest

All three workflows do the summary I posted in the first message, they just build different projects and tests. Workflow1 builds all projects in the solution and runs unit and integration tests, Workflow2 and 3 build just the projects needed to run unit tests and ran those.

ganncamp · July 30, 2024, 11:57am

Hi,

Thanks for that detail. I’ve flagged this for more expert eyes.

Ann

benjamin.raymond · August 2, 2024, 1:36pm

Hello @oana.marina ,

We investigated and created a sample repository to reproduce. We failed to reproduce the issue, the cache was found on successive runs.

Could you please verify that the steps Post Cache SonarQube scanner and Post Cache SonarQube packages are executed after the build step, and could you please send the logs for those two steps?

Could you also ensure the cache entries are present in the GitHub web interface following these steps?

Hope this helps

oana.marina · August 6, 2024, 2:26am

For Workflow1 the Post Cache steps are not executed and nothing is in the Cache.
For Workflow2 the Post Cache steps are not executed and nothing is in the Cache.
For Workflow 3 the Post Cache steps are executed and there are entries in the Cache.

And while looking for the above data I realized the cache is saved when the Sonar Analysis step is successful. For Workflows 1 and 2 that never happens as it analyzes the main branch which is over 200k lines and the steps times out waiting for the quality gate to finish. Do I have to disable the timeout?

benjamin.raymond · August 6, 2024, 7:48am

Hi @oana.marina ,

Yes, actions/cache will only save the cache when the pipeline is successful. Therefore, if having 200k lines is a valid use-case for you and the scan times out, I would suggest increasing the timeout

Bachri_Abdel · August 6, 2024, 8:13am

Hi @oana.marina ,
It may be interesting to promote SonarQube to IT department. I am convinced that the scanner helps to improve the security of your software.
Also it is not safer to download and run the executable in cache.

system · August 13, 2024, 8:14am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.