Report size too big after upgrade from 10.8 to 2025.1

SonarQube Server / Community Build
1

Deployment: Virtual Machine, Alma Linux 8.10, 32 GB ram
Version: Enterprise Edition v2025.1 (102418)

Upgraded our staging Enterprise server from v10.8 (100206) to v2025.1 (102418) and ran sonar analysis on continuous integration infrastructure and it failed to save the reported results (scan succeeded). This appears to be the same as problems reported at…

Between these releases our report size increased by around 100 MB scanning the same code base and now exceeds the limits of the Postgres database. I collected data from our production servers which is still running version 10.8 and compare it to different changes on the 2025.1 version to understand the impact of different changes to get analysis working again.

image
image1689×284 26.1 KB

We have moved to ‘clean as you code’ model and use pull request analysis to gate commits to the monolithic repository that no new sonar issues are being introduced. Although breaking the sonar project down into smaller pieces fixes this report generation issue, this is not viable as developers work in all areas of the monolithic repository and validation of changes using pull request analysis to multiple sonar projects would be very difficult to implement.

Are there other alternatives for solving this issue? This issue is blocking upgrade to 2025.1 for our production server.

Other items of note that relates to this issue. We use our own quality profile (not “Sonar Way”) so there was no change in the rules applied during analysis. There were 8 rules added in the new release, and these new rules did make the problem considerably worse as well as increase scanning time significantly, but this is a separate issue.

Here is the error reported on the sonar enterprise servers when the saving the report fails.

Error Details: ENC [Project Analysis]
Error Details
java.lang.IllegalStateException: Fail to select data of CE task fe236d98-1946-45e8-bdc1-64daca360cc2
                at org.sonar.db.ce.CeTaskInputDao.selectData(CeTaskInputDao.java:74)
                at org.sonar.ce.task.projectanalysis.step.ExtractReportStep.execute(ExtractReportStep.java:67)
                at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:79)
                at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:70)
                at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:57)
                at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
                at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
                at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
                at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
                at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
                at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
                at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
                at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
                at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
                at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
                at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
                at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
                at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
                at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
                at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
                at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.postgresql.util.PSQLException: ERROR: invalid memory alloc request size 1161851657
                at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2733)
                at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2420)
                at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:372)
                at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:517)
                at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:434)
                at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:194)
                at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:137)
                at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
                at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
                at org.sonar.db.ce.CeTaskInputDao.selectData(CeTaskInputDao.java:67)
                ... 20 more
2

Hi,

Are you running 3rd-party plugins? We’ve recently become aware that some (I think it was Dependency Check) store very large measure values. It’s not clear to me whether that’s generated on the CI-side and included in the analysis report submitted to the server, or on the server side, but that’s IMO the first thing to look at.

Beyond that… ~3.5M LOC is a lot. I know you’ve said splitting the project isn’t feasible long-term, but I’m not sure there’s another solution.

 
Ann

3

We do have 5 custom rules that we maintain this is part our quality profile. Here is the breakdown. Other than our rules all other rules I believe are inherited from the sonar way profile and no other rules have been added.

  • 510 Sonar Java Rules
  • 24 Sonar Security Java Rules
  • 6 Dafaflow Bug Detection SonarAnalyzer
  • 5 Custom Java Rules

I removed these custom rules from our quality profile and ran analysis on the 2025.1 staging server. There was not really any significant change in result. I ran with the 700k fewer source files and the baseline result (the last row in the table above) and see these values.

  • Report compression time = 72,272ms
  • Report Size = 493.9 MB
  • Scan Time = 1h 33m
  • Issues Reported = 175,158
  • LOC reported = 2,776,965

Removing the 5 custom rules from the quality profile did not really make a significant difference to report size so I assume this is not the cause of the problem.

4

Hi,

I think that’s a fair assumption, and thanks for stepping through the test.

To be honest, I’m not sure there’s anything to be done here in the short term except what you’ve already identified: split the analysis into multiple projects.

In the medium to long term… well, I’m not sure much will be done there. Changing this would require a fundamental re-architecture, and that would require more “traction” than this topic has. That said, I am going to raise this internally. But I’m not going to hold my breath.

 
Ann

8

Hi again,

As a followup, it turns out you’re less unique in this problem than I thought. Not enough to make it urgent, or anything :sweat_smile: but we have identified a tactic on our side (still nothing you can do, unfortunately) to buy some wiggle room (SONAR-21454). Now it’s “just” a question of getting it un-canceled and prioritized.

 
Ann