I noticed an issue where our cfamily cache would be invalidated randomly, causing hugely costly build processes in terms of time. I saw on this post that the full checkout source path invalidates the cache if changed.
Is this true? Is there a way to stop this? Our CI system creates a workspace folder, e.g. C:\agent_work\1…, C:\agent_work\2…, etc. This means the cache will constantly get invalidated when using a different agent or different workspace number. For reference, we use Azure DevOps (VSTS), but building on jenkins this would also be an issue.
Looking for any ideas for workarounds here, otherwise I will probably have to pull the PR analysis out, and the caching again.
I’ve tried configuring azure devops to use a static sources directory, however this is not really possible.
There is no configurable way to change the “Build.SourcesDirectory” variable. Even if I called git checkout manually to a static location, the “SonarQubeAnalyze” task cant find anything, as it defaults the root directory to Build.SourcesDirectory, meaning no files get scanned:
WARNING: File 'C:\static-agent\myfile.h' is not located under the root directory 'C:\agent\_work\8\s' and will not be analyzed.
And there is no way to set the working directory of that task.
Yes, change of checkout absolute paths between runs invalidate the cache. Out of curiosity, what cache mechanism are you trying to use on Azure devops? Because if you are trying to use Pipeline caching - Azure Pipelines | Microsoft Learn it is not going to work, in that page you can read:
This means subsequent builds for the same branch will not be able to update the cache even if the cache's contents have changed.
Im just using artifacts to store the sonar cache at the end of each run, and grab the latest cache for that branch at the start of each run
I mean configuring the code to be checked out to the same location on our build agents each time, e.g. azure devops checkout task goes to:
C:\agent_work\5(Can be any number)\s
I am trying to configure azure devops to checkout instead to C:\sources, so that the cache does not become invalidated when using a different agent from our build pool. However, checkout location isnt modifiable so the only options are to checkout manually via commandline, or (which I am trying now) move the code after checkout, which also comes with issues of its own. Also worth mentioning doing this stop Azure Devops automatic cleanup operations, as source files are no longer where they expect them to be
Just wanted to say I did get this working eventually, its not a clean solution but it works. However I think its clear that this isnt a smooth onboarding. Also, as soon as I got it working, I have hit the following issue:
This has rendered the feature a bit useless for our use case unfortunately.
For reference for others:
steps:
- checkout: self
clean: true
- powershell: |
# move all your code
Get-ChildItem -Path $env:BUILD_SOURCESDIRECTORY -Force | ForEach-Object { Move-Item -Path $_.FullName -Destination ('C:\staticAgentPath\' + $_.Name) }
- task: SonarQubePrepare@4
inputs:
extraProperties: |
sonar.projectBaseDir=C:\staticAgentPath\
# and the rest
- powershell: |
# modify $env:AGENT_BUILDDIRECTORY\.sonarqube\conf\SonarQubeAnalysisConfig.xml
# add sonar.projectBaseDir property to the "LocalSettings" node
# change AnalysisConfig.SourcesDirectory to C:\staticAgentPath\ also
- powershell: |
# build
workingDirectory: 'C:\staticAgentPath\'
- task: SonarQubeAnalyze@4
- task: SonarQubePublish@4