Sonarqube Scan increased from seconds to hours after upgrade

[cristian_lazaro]

Hello SonarQube Team.

After upgrading from 8.9.4 Enterprise Edition to 9.4 Enterprise Edition, almost all the Scans takes a lot longer than before.
Scans that took between 15-30 seconds, now takes more than 30 minutes (sometimes more than an hour).

Our Sonarqube VM has 64GB of RAM, and our PSQL has also plenty of free memory.
After the upgrade, we performed the “VACUUM FULL” suggestion on the upgrade guide here: Upgrade Guide | SonarQube Docs but the results are the same…

After changing the logs to DEBUG, this is what it shows:

2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] Execution time for each component visitor:
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - AnalysisFromSonarQube94Visitor | time=1ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - LoadComponentUuidsHavingOpenIssuesVisitor | time=26ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - IntegrateIssuesVisitor | time=1312988ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - CloseIssuesOnRemovedComponentsVisitor | time=0ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - MaintainabilityMeasuresVisitor | time=154ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - NewMaintainabilityMeasuresVisitor | time=141ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - ReliabilityAndSecurityRatingMeasuresVisitor | time=54ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - NewReliabilityAndSecurityRatingMeasuresVisitor | time=83ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - SecurityReviewMeasuresVisitor | time=25ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - NewSecurityReviewMeasuresVisitor | time=24ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - LastCommitVisitor | time=18ms
2022.11.28 14:10:15 DEBUG ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.p.s.ExecuteVisitorsStep] - MeasureComputersVisitor | time=22ms
2022.11.28 14:10:15 INFO ce[AYS-R7fgBX4fza9Oibng][o.s.c.t.s.ComputationStepExecutor] Execute component visitors | status=SUCCESS | time=1313716ms

being the IntegrateIssuesVisitor the one that takes 99% of the time.

Do you guys have any suggestion as what else might be missing here so the scans go back to last what they used to?

Thank you very much for your time.

Regards, Cristian.

[Colin]

Hey there.

SonarQube v9.4 is already several versions behind (and EOL). It’s not an LTS version – which means you should move to v9.7 as soon as possible (v9.8 due out next week or the week after).

I would also recommend performing a VACUUM FULL ANALYZE

SonarQube becomes slow after PostgreSQL upgrade

Hello SonarQube Operators! It’s common that after performing an upgrade of PostgreSQL, a SonarQube instance connected to a PostgreSQL database can become painfully slow. Thankfully, we know how the issue can usually be addressed (it works almost every time!): by executing the following command against your Postgres database. VACUUM FULL ANALYZE For a PostgreSQL database, a VACUUM FULL command reclaims unused storage, while the ANALYZE command updates statistics used by the planner to determi…

[cristian_lazaro]

Hi @Colin
Thanks for your suggestions.
We will upgrade our Sonarqube version to the last one (9.7.1.62043) and perform the VACUUM FULL ANALYZE hoping this will fix our timing issues.

Thanks again!

[colinmcd (Colin)]

Were having this same issue.
Scans go from 1-3 minutes to 30 minutes after migrating the database (postgres 9.6 to postgres 11).
We have run VACUUM FULL ANALYZE Is there any test queries that we can bench?

If we had a slow query we could bench we can then verify that VACUUM FULL ANALYZE worked or if another problem is effecting us.

[nshulman (Nina)]

We are having the very same issue. We need to upgrade from Sonar 9.5 to 9.8 Developer edition and it requires to migrate sonarDB from postgres 9.6 to postgres11, so we are trying to do db migration and test if that works with existing SQ 9.5. After migration, SQ comes ups and we can see all the projects, but every scan takes extra long and times out.

We have run VACUUM FULL ANALYZE, but not sure how to verify that it was successful and if we not having different kind of issue causing scans taking much longer and failing with a timeout error.

Please advice on how to verify VACUUM FULL ANALYZE success and/or what else to try.

[colinmcd (Colin)]

A postgresql Gotcha

Do run Vacuum full analyze on each table, not the database. Record the results. If a table is particularly long rerun vacuum full analyze to confirm that it vacuums down.

Another problem is if you use Postgres with a distributed service like Patroni, that seems to confuse sonarqube in version 9.5

[Colin]

@colinmcd

Do run Vacuum full analyze on each table, not the database. Record the results. If a table is particularly long rerun vacuum full analyze to confirm that it vacuums down.

A user also shared a script to accomplish this here:

Issues panel not working after upgrade

Hi, It’s fixed, thanks ! VACUUM FULL ANALYZE + re-indexation did the job. We executed the following command, as postgres user, to execute it on all the tables: for TABLE in $(psql -d sonar -c "\dt" | grep table | awk -F "|" '{print $2}' | tr -d " "); do psql -d sonar -c "VACUUM FULL ANALYZE $TABLE" ; done