SonarQube Developer Edition 10.3 Error 500 Uploading Report

ERROR: Error during SonarScanner execution
6247java.lang.IllegalStateException: Failed to upload report: Error 500 on https://a2i2-sq.761link.net/api/ce/submit?projectKey=eot_mav_platform_r2_AY021p1jTVMTeanOUF7B : {“errors”:[{“msg”:“Fail to insert data of CE task AY_zpnDDOA8cMBnzxBVF”}]}

Caused by: org.sonarqube.ws.client.HttpException: Error 500 on https://a2i2-sq.761link.net/api/ce/submit?projectKey=eot_mav_platform_r2_AY021p1jTVMTeanOUF7B : {“errors”:[{“msg”:“Fail to insert data of CE task AY_zpnDDOA8cMBnzxBVF”}]}

Our Sonarqube application is deployed using docker and is connected to our Gitlab postgresql database. We’re guessing the massive report getting sent up to the server is getting bounced for being too big for the default message size allowance on the server. The report is about 2.7GB.

The changes made on database postgresql.conf file were:

shared_buffer parameter from 128MB to 256MB
uncommented the parameter work_mem 4MB

Changes made on sonar.properties (application config file):

Update the min and max for web and ce from their defaults to 4096
sonar.web.javaOpts=-Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError
sonar.ce.javaOpts=-Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError

included:
sonar.web.maxContentLength=5000000000
sonar.web.ws.timeout=300

Still receiving the error with updated changes listed above. Appreciate any help! Thank you.

Hey there.

  • Can you check your ce.log for a more detailed error message?
  • 2.7 GB is a huge analysis report! What kind of project is this? How many files are indexed by the scanner?

Hi Colin,

We’ve pared this down pretty far actually, though the repo we’re drawing from is for a large project. The analysis file I most recently tried to send up was about 250MB, though compression probably brought that down even more. We still got the 500 error from the server.

It’s certainly still possible at 250 megabytes we’d still be tripping some cap on the server or db level for the file size, though it’s hard to say for sure that’s what’s happening. Like Lauren said, we made those above changes to the postgress db settings, and the sonar server’s config file, but that didn’t shake anything loose. I don’t have access to that ce.log file, but Lauren I believe you said earlier that it didn’t show much of anything at the time of the upload? If there’s anything else we could look at, Colin, in terms of logging, please let us know.

Im just seeing lines about the maxLifeTime value:


2024.07.09 18:50:07 INFO  ce[][o.s.c.c.ComputeEngineContainerImpl] Running Developer edition
2024.07.09 18:50:07 INFO  ce[][o.s.ce.app.CeServer] Compute Engine is started
2024.07.09 22:29:18 WARN  ce[][c.z.h.pool.PoolBase] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@176e4aae (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024.07.09 22:29:18 WARN  ce[][c.z.h.pool.PoolBase] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@254f90da (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024.07.09 22:29:18 WARN  ce[][c.z.h.pool.PoolBase] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@55e12e1e (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024.07.09 22:29:18 WARN  ce[][c.z.h.pool.PoolBase] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@303f2536 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024.07.09 22:29:18 WARN  ce[][c.z.h.pool.PoolBase] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@15ea62e7 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024.07.09 22:29:18 WARN  ce[][c.z.h.pool.PoolBase] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@4c7ae9c0 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024.07.09 22:29:26 ERROR ce[][o.s.c.t.CeWorkerImpl] Failed to pop the queue of analysis reports
org.apache.ibatis.exceptions.PersistenceException:
### Error querying database.  Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 8001ms.
### The error may exist in org.sonar.db.property.InternalPropertiesMapper
### The error may involve org.sonar.db.property.InternalPropertiesMapper.selectAsText
### The error occurred while executing a query
### Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 8001ms.
        at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:156)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:142)
        at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147)
        at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80)
        at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:142)
        at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86)
        at jdk.proxy2/jdk.proxy2.$Proxy53.selectAsText(Unknown Source)
        at org.sonar.db.property.InternalPropertiesDao.selectByKey(InternalPropertiesDao.java:179)
        at org.sonar.ce.queue.CeQueueImpl.getWorkersPauseStatus(CeQueueImpl.java:350)
        at org.sonar.ce.queue.InternalCeQueueImpl.peek(InternalCeQueueImpl.java:78)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.tryAndFindTaskToExecute(CeWorkerImpl.java:170)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:153)
        at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

Hey there.

This all reminds me of an issue that was fixed in SonarQube v10.4.1 (SONAR-21655).

The issue only impacted SonarQube v10.4 (not v10.3). Can you confirm it’s v10.3 in your environment and not v10.4?

In any case – you should know that SonarQube v10.3 is EOL, and the currently active v10.x version is v10.6.

Yes, confirmed we’re on version 10.3.0.82913. We’ll work on upgrading to the latest 10.6.x once approved and I’ll report back when complete. Thanks!