Compute Engine not finishing a task

[msemich (Matt Semich)]

Must-share information (formatted with Markdown):

• Version: Enterprise Edition v10.7 (96327)
• Installation: zip installed on a RHEL8 VM PostgreSQL 15.8
• Goal: Trying to get Compute Engine to process an analysis report generated from sonar-scanner 5.0.1.3006. We were getting a JavaHeapDump until we increased the RAM to 10GB.
• What have we tried so far: Increased RAM and Cores for Compute Engine and turned on DEBUG levels for Compute Engine.

Below is the Compute Engine log file for the task associated to the issue:

DEBUG ce[task-id][o.s.c.t.p.s.ExtractReportStep] Analysis report is 550.2 MB uncompressed
INFO  ce[task-id][o.s.c.t.p.s.ComputationStepExecutor] Extract report | status=SUCCESS | time=2203ms
INFO  ce[task-id][o.s.c.t.p.s.ComputationStepExecutor] Persist scanner context | status=SUCCESS | time=9ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Propagate analysis warnings from scanner report | status=SUCCESS | time=0ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Generate analysis UUID | status=SUCCESS | time=0ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load analysis metadata | status=SUCCESS | time=3ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Initialize | status=SUCCESS | time=2ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Build tree of components | components=130 | status=SUCCESS | time=16ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Validate project | status=SUCCESS | time=1ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load quality profiles | status=SUCCESS | time=1257ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Checks executed before computation of measures | status=SUCCESS | time=2ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Generate SQ Upgrade analysis events | status=SUCCESS | time=1ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load file hashes and statuses | status=SUCCESS | time=38ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load Quality gate | status=SUCCESS | time=27ms
DEBUG ce[task-id][o.s.c.t.p.p.NewCodePeriodResolver] Resolving first analysis as new code period as there is only one existing version
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load new code period | status=SUCCESS | time=28ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load prioritized rules | status=SUCCESS | time=10ms
DEBUG ce[task-id][o.s.c.t.p.f.FileMoveDetectionStep] Either no files added or no files removed. Do nothing.
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Detect file moves | reportFiles=127 | dbFiles=124 | addedFiles=3 | status=SUCCESS | time=6ms
DEBUG ce[task-id][o.s.c.t.p.f.PullRequestFileMoveDetectionStep] Currently not within Pull Request scope. Do nothing.
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Detect files moves in Pull Request scope | status=SUCCESS | time=0ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load duplications | duplications=356 | status=SUCCESS | time=45ms
DEBUG ce[task-id][o.s.c.t.p.d.CrossProjectDuplicationStatusHolderImpl] Cross project duplication is disabled because it's disabled in the analysis report
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute cross proejct duplications | status=SUCCESS | time=0ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute size measures | status=SUCCESS | time=38ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute new coverage | status=SUCCESS | time=639ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute coverage measures | status=SUCCESS | time=12ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute comment measures | status=SUCCESS | time=1ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute duplication measures | status=SUCCESS | time=121ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute size measures on new code | status=SUCCESS | time=42ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute language distribution | status=SUCCESS | time=10ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute test measures | status=SUCCESS | time=1ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute complexity measures | status=SUCCESS | time=1ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Load measure computers | status=SUCCESS | time=0ms
INFO  ce[task-id][o.s.c.t.s.ComputationStepExecutor] Compute Quality Profile status | status=SUCCESS | time=12ms
DEBUG ce[task-id][c.z.h.p.ProxyConnection] HikariPool-1 Executed rollback on connection org.postgresql.jdbc.PgConnection@<number> due to dirty commit state on close().
DEBUG ce[task-id][c.z.h.pool.PoolBase] HikariPool-1 Reset (autoCommit) on connection org.postgresql.jdbc.PgConnection<number>

[Colin]

Hey there.

• Does the Compute Engine task appear as Cancelled in the UI, or does it continue forever?
• Does SonarQube process any tasks, or is it a problem sporadically / with a certain project / something else?

[msemich (Matt Semich)]

It finished after almost 24 hours of executing. This one project is from auto generated code and the same issue is marked for a lot of the lines of code. We are talking about 2.2 million issues. I ran it again after the initial run and it completed within 3 minutes. Do you have any suggestions on how to handle large code bases for the initial analysis? Or just wait until it finishes?

[Colin]

That’s definitely too long. A few things to consider:

• I think it would be interesting to see your complete ce.log for this task now that it’s finished to see where the hold-up was
• We discourage analyzing generated code because it usually isn’t that useful to users – are your developers able to go fix the issues in that code? No? Then… analyzing that code with SonarQube won’t do you much good.