We have started using the new caching feature provided by the latest SonarCFamily plugin with our on-prem SonarQube instance and have been very impressed with the performance benefits it provides, particularly with regards to improving PR analysis times on our larger projects. One question I did have was what will cause a cache to be invalidated i.e. it will no longer be able to be used and a full analysis will again be required? For example, would any of the following invalidate the cache:
Upgrading SonarQube to new version
Upgrading the SonarCFamily plugin to new version
Changing the C++ Quality Profile used by the project
Changing the Quality Gate used by the project
It would be good to have a clear list, as it would allow us to plan accordingly for actions that would invalidate our caches. That in turn could impact PR analysis on large projects with lengthy analyses where there would no longer be a valid cache for the target branch until it is re-analysed.
in general you shouldn’t care too much about cache invalidation as it is implementation detail and it could change in the future. In any case this is a list of things (more or less exhaustive) invalidating the cache:
Upgrade/Downgrade of SonarQube version if it comes with different version of SonarCFamily plugin
Upgrade/Downgrade of SonarCFamily plugin
Change of Quality Profile
Change of compiler configuration (include dirs, macros, etc…)
Change of source checkout filesystem location (there is a dependency on absolute paths)
I just wanted to ask for clarification on a slightly more detailed version of the scenario outlined in this thread. Basically in our CI environment we too have ephemeral machines, so in a simple overview we do the following:
For master branch builds we download the remote cache for that branch (stored in Artifactory) prior to the analysis and update the remote cache after the analysis has completed.
For PR branch builds we download a remote cache for the master (i.e target) branch prior to the analysis and the remote cache is not updated following the analysis.
This all works well for the most part, but there are some caveats. We run an on-prem instance of SonarQube and analysis of our larger projects can take 1-3 hours, which of course makes the cache very useful! However we are seeing the following pattern occur:
A developer raises a PR which includes changes that cause the cache from the target branch to be invalidated for whatever reason e.g. the include directories are change due to a dependency being modified etc. (see SonarCFamily Cache Question that I raised earlier)
This then means the full lengthy analysis occurs.
In the meantime, feedback on the changes in the PR requires the developer to make further commits.
This then causes the analysis to be run again on the subsequent PR builds, each with the lengthy full analysis.
Is there anything you can suggest that may let us leverage the cache in this situation? I understand that the full analysis must occur for the first PR build, but would it be advisable after that first build to create a (remote) copy of the PR’s cache and use that for the subsequent commits (if there are any)? Would that have any effect on what issues are reported against the PR in SonarQube?
in general there is no best rule for cache, it really depends from your use case.
For instance, you could try to evaluate if it makes sense to save cache for PRs too, for instance for PRs you can initially fetch master branch for the first analysis and then you keep each PR cache for second analysis. I think that you can set automatic purge of artifacts on Artifactory to automatically purge PRs cache.
Sorry for the late reply. Just wanted to say thanks for the feedback, I’ve done some testing and subsequently have implemented your suggestion. Hopefully we’ll get some positive feedback from development teams over the coming weeks.