Housekeeping / Database Cleaner / version event handling

sonarqube
(Günter Wirth) #1

Hi,

We are using version numbers with 4 digits, the last number is the build number. Forwarding this to SQ creates in bigger teams typically more than one version event per day. As a result housekeeping is not working (https://docs.sonarqube.org/7.4/instance-administration/housekeeping/) because snapshots have version events assigned.

On the same day this version number with build number is very useful. But after a day/week, … it could be ignored.

  • Would be good if feature works (clean-up) existing projects.
  • Different projects have different version number formats. So maybe you can provide a filter to define the “long-living” part of the version string.
  • Or simple flag: Keep only one version per day / week / month / year, …

see also: Housekeeping / version event

Regards,
Günter

(G Ann Campbell) #2

Hi Guenter,

You should actually see some progress on this in 7.7. What I anticipate is that you’ll need to stop passing your full semantic version as sonar.projectVersion (for the reasons you’ve detailed) and use instead a different parameter we’ll provide. Note that you’ll need to manually clean out the old analyses with build-string-version-events.

 
Ann

1 Like
(Gilbert Rebhan) #3

Hi Ann ,

which parameter do you propose ? For many legacy projects we’re using sonar.projectVersion for every scan, so we will get problems ? We’ve used this since starting with Enterprise edition 5.x in 2017 … currently running 7.6 in production and i didn’t notice any problems with that.

Cheers,
Gilbert

(G Ann Campbell) #4

Hi Gilbert,

This isn’t available yet; it (SONAR-11631) will (probably) show up in 7.7. (Check the release notes; it will be close.) What you would do is pass your full semantic build string to the new parameter, and use sonar.projectVersion only for the “big” version. For instance:

  • semantic version 7.7.0.483
    sonar.buildString=7.7.0.483
    sonar.projectVersion = 7.7.0
  • semantic version 7.7.0.484
    sonar.buildString=7.7.0.484
    sonar.projectVersion = 7.7.0
  • semantic version 7.8.0.485
    sonar.buildString=7.8.0.485
    sonar.projectVersion = 7.8.0

Regarding your question about having used sonar.projectVersion since 2017, that’s normal and expected, and if you’ve only been using it to pass the ‘big’ version, then you won’t have any problems at all. If you’ve been using it to pass the full, semantic version, then you may be experiencing some database bloat and should consider some manual housekeeping.

 
HTH,
Ann

(Gilbert Rebhan) #5

Hi Ann,

thanks for the details. In fact for those legacy projects, every scan is a big version, there are no snapshots or in between versions. So i guess we might get problems eventually.
What are typical indicators for problems, where should i look for manual housekeeping ?
(we’re using MSSQL 2014)
WRT to the new sonar.buildString , i don’t yet understand the reasons / adavantage. After all it’s just another parameter that has also to be persisted / stored in DB.
What’s the difference to using sonar.projectVersion=1.0-SNAPSHOT several times until the next release build which gets version 1.0 ? OK, if using the buildString as you proposed in your example there is an unique identifier for every scan. But doesn’t that work against housekeeping as i have to persist another parameter ?

Gilbert

(G Ann Campbell) #6

Hi Gilbert,

That’s actually exactly the point of the new parameter (which will be included in 7.7). There are people who for traceability reasons need to have the build number (full, semantic version string) associated with the analysis. So to date, they’ve been using that as the sonar.projectVersion and then having to perform housekeeping manually because each analysis with a sonar.projectVersion different from the previous one is stamped with a Version Event and by default those analyses are exempt from housekeeping for five years. And without housekeeping, you run the risk of database bloat.

With the new parameter, those people can both store the semantic version in a retrievable way and have housekeeping work properly.

But it sounds like that’s not an issue for you.

 
Ann

(Günter Wirth) #7

Hello Ann,

we tried to manually clean-up our MySql database. We replaced strings in the MySql database of the version events with an unique one, e.g. instead of 1.2.3.4, 1.2.3.5, 1.2.3.6, … it’s now all 1.2.3. After that we started a new analysis. Clean-up is still not working. (We tried it with 6.7 to prepare an update to next LTS)

Should this work or do we have to delete the version events?

If we have to delete it I wonder How this could work with 7.7?

sonar.buildString=7.7.0.483
sonar.projectVersion=7.7.0

…because there are also always version events with every analysis? Or do you add no new event if version number is the same as with the last snapshot?

Regards,

(G Ann Campbell) #8

Hi Guenter,

First, I’m required to start by telling you not to update the DB contents directly. It should be treated like a black box. Those version string changes can and should be made through the UI instead.

With or without the changes in 7.7, analyses with version events are going to be exempt from housekeeping, regardless of the format of the version strings. The point of the additions in 7.7 is that you can now use an extra parameter (sonar.buildString) to set build-specific strings without setting a version event.

So to answer your question, what you want to do is:

  • manually remove any unwanted version events (you could also manually delete the extra analyses at this point)
  • update your analysis configuration to pass sonar.buildString and sonar.projectVersion as you’ve outlined

And then, yes, version events will only be added when the value of sonar.projectVersion changes, and the value of sonar.buildString will be stored with the analysis for later use. BTW, right now you can only see that value in the analysis search results. After the LTS we plan to make it available via the UI as well.

 
HTH,
Ann