The “counters” are calculated and stored metric values (i.e. “measures”). They come straight from the DB. I believe your problem is the Elasticsearch indices, which you lose when you reset the data directory. Maybe let them regenerate and then hold on to them?
It takes a while to rebuild those Elasticsearch indices, but SonarQube has progressive availability. That means that once a project is fully reindexed, its data will show up but other projects may still be queued.
What I meant was that normally no important information is stored in the data directory. This directory should only contain the files for elasticsearch and other temporary files or an H2 database (but I use an external POSGRESQL database). When I recreate or upgrade my container, elasticsearch rebuilds itself. I see data reload spots in every project.
Some projects have their tickets going up, others don’t. And even after waiting several hours.
I don’t see how adding a volume for data will solve my problem.
In the official documentation, using a volume is just recommended, not mandatory. I accept having a slower sonarqube at startup, but not losing data.
For information, as mentioned, I am using version 9.9 LTS enterprise and I was able to reproduce the problem.
I also looked in the database to see if I had any jobs in fails or queues but I have nothing.
I confirm that this behaviour is mostly caused by ElasticSearch data/es folder to be out of sync. Reindexation (or Data Reload) is run everytime SonarQube detect that the indexes are missing at startup in the data/es folder, and push the data on ES from what we have on the database.
To confirm that it’s an indexation issue, can you please try to reindex the issues for one of the problematic project? You can use this API:
This will create a new background task to reindex this specific project. Once the tasks is processed, you should see the data updated.
This endpoint goal is to allow SQ admins to reindex a project individually, the main use case we had in mind being after a full reindexation, if you have some project in error, you can reindex them without triggering a full reindexation. It is not designed to reindex all issues. You can do it, sure, it will work, but you should not need it.
Let’s dig into your docker configuration and usage, as it is most likely the root cause of the problem. Could you please describe your docker setup, how it’s configured?
I’m going to tag an Ops from the team to investigate on your docker configuration side
Hello @ktibi, thanks a lot for taking the time to participate in the community.
That terraform module is not managed by SonarSource and is therefore not officially supported.
Nonetheless, i want to help you on that topic, could you provide me some more information ?
What is the lifecycle of that docker container ? In your fargate setup, can you ensure the container is properly killed before starting a new one ?
Also when you say
and i reset every time the /opt/sonarqube/data
Do you take action to delete that one, or do you mean that after each creation that folder is empty ?
Just to clarify a bit, the persistency here is not required, nonetheless you must ensure that only one instance of SQ is connected to the database at anytime, and that when the container start, the folder is empty.