eraonel
October 8, 2024, 1:16pm
1
Hi,
We are using
Community Edition
Version 9.9.2 (build 77730)
We noticed that db sonarqube uses create large files that takes up a lot of space.
Se below.
sudo find /var/lib/pgsql -type f -size +100M -exec ls -lh {} ; | awk ‘{ print $9 ": " $5 }’
I ran the postgresql command:
VACCUM FULL;
But it does not seem to make the usage of diskspace smaller.
br,
//mike
eraonel
October 9, 2024, 12:26pm
2
Bump! Friendly reminder. Files are growing and eating up our disk. How do I know what to clean?
ganncamp
October 9, 2024, 1:46pm
3
Hi,
SonarQube’s housekeeping runs automatically on each project after a branch is analyzed.
I would not recommend manually deleting any of the files you’ve listed. That sounds like a good way to break things.
Can you provide the sizes of your largest DB tables?
eraonel
October 10, 2024, 5:36am
5
Here is form sonarqube db postgresql 12:
table_name | pg_size_pretty | pg_total_relation_size
---------------------------+----------------+------------------------
live_measures | 15 GB | 16351313920
file_sources | 11 GB | 11960778752
project_measures | 1250 MB | 1310687232
components | 840 MB | 881229824
issues | 446 MB | 467542016
issue_changes | 241 MB | 252485632
analysis_properties | 35 MB | 36331520
ce_scanner_context | 25 MB | 25829376
snapshots | 19 MB | 20332544
ce_activity | 18 MB | 19079168
events | 18 MB | 18653184
project_branches | 18 MB | 18522112
ce_task_message | 15 MB | 15294464
ce_task_characteristics | 8016 kB | 8208384
rule_desc_sections | 7448 kB | 7626752
scanner_analysis_cache | 3736 kB | 3825664
rules | 2752 kB | 2818048
ce_task_input | 1832 kB | 1875968
qprofile_changes | 1816 kB | 1859584
group_roles | 1224 kB | 1253376
user_roles | 904 kB | 925696
active_rules | 568 kB | 581632
project_links | 456 kB | 466944
projects | 392 kB | 401408
deprecated_rule_keys | 264 kB | 270336
rules_parameters | 248 kB | 253952
org_qprofiles | 192 kB | 196608
push_events | 184 kB | 188416
users | 168 kB | 172032
rules_profiles | 160 kB | 163840
metrics | 128 kB | 131072
groups_users | 112 kB | 114688
active_rule_parameters | 96 kB | 98304
user_tokens | 96 kB | 98304
properties | 88 kB | 90112
groups | 80 kB | 81920
plugins | 80 kB | 81920
project_qgates | 72 kB | 73728
ce_queue | 72 kB | 73728
default_qprofiles | 72 kB | 73728
project_qprofiles | 72 kB | 73728
qprofile_edit_users | 72 kB | 73728
session_tokens | 72 kB | 73728
project_badge_token | 72 kB | 73728
permission_templates | 64 kB | 65536
internal_properties | 64 kB | 65536
rule_repositories | 64 kB | 65536
perm_templates_groups | 56 kB | 57344
quality_gates | 56 kB | 57344
quality_gate_conditions | 56 kB | 57344
perm_templates_users | 56 kB | 57344
es_queue | 48 kB | 49152
app_branch_project_branch | 48 kB | 49152
user_dismissed_messages | 40 kB | 40960
schema_migrations | 40 kB | 40960
notifications | 40 kB | 40960
project_alm_settings | 40 kB | 40960
webhook_deliveries | 40 kB | 40960
event_component_changes | 40 kB | 40960
app_projects | 32 kB | 32768
project_mappings | 32 kB | 32768
new_code_periods | 32 kB | 32768
duplications_index | 24 kB | 24576
internal_component_props | 24 kB | 24576
portfolios | 24 kB | 24576
alm_pats | 24 kB | 24576
audits | 24 kB | 24576
alm_settings | 24 kB | 24576
qprofile_edit_groups | 24 kB | 24576
portfolio_projects | 16 kB | 16384
new_code_reference_issues | 16 kB | 16384
scim_users | 16 kB | 16384
webhooks | 16 kB | 16384
perm_tpl_characteristics | 16 kB | 16384
qgate_group_permissions | 16 kB | 16384
qgate_user_permissions | 16 kB | 16384
portfolio_references | 16 kB | 16384
saml_message_ids | 16 kB | 16384
portfolio_proj_branches | 8192 bytes | 8192
br,
//mike
ganncamp
October 10, 2024, 12:30pm
6
Hi mike,
I’m not an expert in this area, but nothing sticks out at me here.
Based on the size of file_sources, it looks like you just have a lot under analysis. Is this a large number of projects, or fewer projects with lots of branches? Perhaps you could tweak the inactive branch setting ?
You said you ran a VACUUM FULL. Have you also tried (after a backup) rebuilding the indices and refreshing the statistics?
eraonel
October 10, 2024, 12:37pm
7
Hi Ann,
How can I perform “rebuilding the indices and refreshing the statistics” from sonar or is it something to be done in the db?
//mikael
ganncamp
October 10, 2024, 12:57pm
8
Hi mikael,
That’s DB maintenance done on the database side.
ganncamp
October 14, 2024, 1:11pm
9
Hi again,
You may also be interested in this ticket, which will be part of 10.8:
SONAR-22870 - [10.8] Compress live_measures table
It should help your situation.
We currently expect 10.8 to be announced in early December.