SonarCloud server cache disable due to change in quality profile

Hello !
We are facing cache issues and we would like to improve analysis performance by restoring it.

  • ALM and CI used :
    Azure DevOps

  • Scanner command used when applicable (private details masked)
    Use of default configuration for cache (sonar.cfamily.analysisCache.mode=server)

  • Languages of the repository
    C++

  • Error observed

INFO: Loading cache from: server
INFO: Existing cache invalidated and deleted, change detected in quality profile 

The last quality profile change is 25 days old so I don’t understand why the use of the cache is not working.

We are considering handling the cache locally but we would like to know if there is a way to fix the current behaviour.

Thanks for your help

Hi @AlexJulita, and thanks for looking through the logs and for bringing our attention to this, we appreciate it,

We have recently become aware of an issue, and we have implemented a potential solution in [CPP-5158] - Jira which is going to be released in CFamily plugin version 6.54 this week (expected to be available on SonarCloud sometime next week).

Please let us know if you still encounter these unexpected cache invalidations during the first few weeks following the deployment of CFamily plugin 6.54. We can keep this thread open to monitor the situation.

Thanks,
Michael

2 Likes

Great !
Thank you for your quick answer, we will hold our local cache implementation then, I will post a message on this thread to inform you of our feeling about your fix !

Hi @AlexJulita, and thanks again for looking through the logs and bringing our attention to the issue. As mentioned earlier, we have released the fix and it was deployed less than 2 weeks ago. We have also been monitoring the situation since then, and our impression seems to be good so far. Please let us know if you have observed this unexpected cache invalidation at any time during the last week.

We are also planning to continue monitoring the situation for some time, and we can keep this thread open for a few more weeks…

Best regards,
Michael

Hello !

Some update on the subject; from what I see cache is working sometimes and when it does performances are really great !
For examples :
PR that modifies 21 files :

INFO: Analysis done in: 22516ms
INFO: 1201/1239 files marked as unchanged
INFO: Cache: 581/582 hits, 5676076 bytes
INFO: SE: 582 out of 582

But sometimes (and I feel, most of the time but I didn’t analyzed deeply):
PR that modifies 261 files:

INFO: Analysis done in: 820626ms
INFO: 1198/1239 files marked as unchanged
INFO: Cache: 0/582 hits, 5681794 bytes
INFO: SE: 582 out of 582

PR that modifies 7 files :

INFO: Analysis done in: 702743ms
INFO: 1197/1239 files marked as unchanged
INFO: Cache: 148/582 hits, 5679529 bytes
INFO: SE: 582 out of 582

I can’t explain exactly why, maybe this is related to our software architecture and when some files that are included in a lot of other are modified it cancels the cache hits but in this cas I can’t explain the ‘XX files marked as unchanged’

Maybe we will try to use the local cache to compare the results

Best regards !

Hi @AlexJulita and thanks for sharing the information with us,

The “files marked as unchanged” log entry is related to a separate optimization mechanism that saves the time normally needed to transfer and process the issues reported from these files on the server. Note that this doesn’t say anything about whether the analyzer had to process these files from scratch or load their issues directly from the cache.

Therefore, I would suggest focusing on the number of cache hits to measure how much analysis work the cache saved.

I can’t explain exactly why, maybe this is related to our software architecture and when some files that are included in a lot of other are modified it cancels the cache hits

Indeed, file dependencies and architecture strongly impact cache performance. Generally speaking, the analyzer cache performance should be correlated to the performance of incremental builds after the changes in your PR; If a header is changed, the compiler needs to reprocess all TUs that include this header, and so does the analyzer.

Maybe we will try to use the local cache to compare the results

You can also check the exact reason the cache was not useful for each TU cache miss if you enable debug logs sonar.verbose=true. This should provide useful hints to help improve cache utilization. I would recommend starting with this first, before trying the file system cache.

I hope this helps,

Best regards,
Michael