How to properly configure memory to JVM on SonarQube

Amine_Tanboura · January 25, 2024, 10:23am

Versions SonarQube 10.2.0 , Sonarlint 7.6.0 , Sonarqube Extension for Azure DevOps v5.17.2)
*SonarQube is deployed with zip,
Description :
We are are using SonarQube Enterprise Edition, version : 10.2.0.77647 and
we are facing issues with instability due to Memory saturation of the JVM. In our situation we have 4,778 M LOC with 36 projects onboarded and we are allocating the following values:

# JVM options of Elasticsearch process
sonar.search.javaOpts=-Xmx4G -Xms4G -XX:MaxDirectMemorySize=1G -XX:+HeapDumpOnOutOfMemoryError
# WEB SERVER
sonar.web.javaOpts=-Xmx6G -Xms6G -XX:+HeapDumpOnOutOfMemoryError
# COMPUTE ENGINE
sonar.ce.javaOpts=-Xmx6G -Xms6G -XX:+HeapDumpOnOutOfMemoryError

The largest project has 1.2 M LOC
Store Size is 201.2 MB
How can we calculate the proper value to each one ?

*LOG extraction

2024.01.24 17:37:08 WARN  web[][o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [Thread-9] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.base@17.0.8/sun.nio.ch.SocketDispatcher.read0(Native Method)
 java.base@17.0.8/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:46)
 java.base@17.0.8/sun.nio.ch.NioSocketImpl.tryRead(NioSocketImpl.java:261)
 java.base@17.0.8/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:312)
 java.base@17.0.8/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
 java.base@17.0.8/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
 java.base@17.0.8/java.net.Socket$SocketInputStream.read(Socket.java:966)
 java.base@17.0.8/java.io.BufferedInputStream.fill(BufferedInputStream.java:244)
 java.base@17.0.8/java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
 java.base@17.0.8/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
 java.naming@17.0.8/com.sun.jndi.ldap.Connection.run(Connection.java:856)
 java.base@17.0.8/java.lang.Thread.run(Thread.java:833)

I look forward to hearing from you!
Thank you in advance.

ganncamp · January 26, 2024, 3:20pm

Hi,

Welcome to the community!

The log snippet you’ve provided seems unrelated to this. Can you give more details on this instability?

Thx,
Ann

Amine_Tanboura · January 29, 2024, 8:37am

Hello Ann,

Thank you for your response, would you please check this log then :
2024.01.26 09:46:56 INFO es[o.e.n.Node] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=D:\APP\SonarQube\current\temp, -XX:ErrorFile=D:\APP\SonarQube\current\logs\es_hs_err_pid%p.log, -Xlog:disable, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=D:\APP\SonarQube\current\temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Des.enforce.bootstrap.checks=true, -Xmx7G, -Xms7G, -XX:MaxDirectMemorySize=1G, -XX:+HeapDumpOnOutOfMemoryError, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=D:\APP\SonarQube\current\elasticsearch\lib,
–add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]

Regards,
Amine

ganncamp · January 29, 2024, 1:01pm

Hi Amine,

Again, what you’ve provided seems unrelated.

What are the symptoms of “instability” that you’re experiencing?

Ann

Amine_Tanboura · January 30, 2024, 9:54am

Hello Ann,
Thank you again for your reactivity!
We detect the instability when we have the error page 502 while opening the Sonarqube platform (which means that the server is no longer reachable).
From Azue when try to launch a sonarqube scan we get the following error (see attached)
In server side the JVM exceeds allocated memory limits which is 6gb for each of the three (screenshot below also) but the service is always running
In the log we don’t have a clear error message when sonarqube becomes unreachable.

Fyi we are using the sonarlint plugin.

Regards,
Amine Tanboura

ganncamp · January 30, 2024, 7:23pm

Hi Amine,

The 502 and the SOCKETTIMEOUT in your screenshot both reflect a failure to reach SonarQube. But unless you’re seeing errors in your server logs, I’m going to have to refer you elsewhere for the cause.

More than likely, you have something “helpful” on your network, such as a proxy, that’s interfering. You should talk to your network folks.

HTH,
Ann

MarwaHamdi · February 7, 2024, 10:25am

Hello Ann,

I’m reaching out as a member of the same team as Amine.
We have checked with our network team and they confirmed that our SonarQube instance is not using the proxy. However, we noticed that when SonarQube becomes unreachable, attempting to access localhost:9000 yields no response. Also upon restarting the SonarQube service, we’ve noticed that the JVM for Elasticsearch quickly reaches the heap limit as soon as the service is up. This behavior seems unusual to us.
Could you please provide guidance on troubleshooting steps to diagnose the issue?

Thank you for your help.

Regards,
Marwa Hamdi

ganncamp · February 7, 2024, 2:08pm

Hi Marwa,

Welcome to the community!

In my previous response I referenced checking the server logs. Have you done that?

Ann

MarwaHamdi · February 9, 2024, 9:06am

Hello,
Yes we checked the server logs this is what we have so far

2024.02.08 15:17:09 WARN  web[][o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [Thread-3248] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:\n java.base@17.0.8/sun.nio.ch.SocketDispatcher.read0(Native Method)\n java.base@17.0.8/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:46)\n java.base@17.0.8/sun.nio.ch.NioSocketImpl.tryRead(NioSocketImpl.java:261)\n java.base@17.0.8/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:312)\n java.base@17.0.8/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)\n java.base@17.0.8/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)\n java.base@17.0.8/java.net.Socket$SocketInputStream.read(Socket.java:966)\n java.base@17.0.8/java.io.BufferedInputStream.fill(BufferedInputStream.java:244)\n java.base@17.0.8/java.io.BufferedInputStream.read1(BufferedInputStream.java:284)\n java.base@17.0.8/java.io.BufferedInputStream.read(BufferedInputStream.java:343)\n java.naming@17.0.8/com.sun.jndi.ldap.Connection.run(Connection.java:856)\n java.base@17.0.8/java.lang.Thread.run(Thread.java:833)
2024.02.08 15:17:09 WARN  web[][o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [OkHttp TaskRunner] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:\n java.base@17.0.8/java.lang.Object.wait(Native Method)\n java.base@17.0.8/java.lang.Object.wait(Object.java:472)\n app//okhttp3.internal.concurrent.TaskRunner$RealBackend.coordinatorWait(TaskRunner.kt:294)\n app//okhttp3.internal.concurrent.TaskRunner.awaitTaskToRun(TaskRunner.kt:218)\n app//okhttp3.internal.concurrent.TaskRunner$runnable$1.run(TaskRunner.kt:59)\n java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n java.base@17.0.8/java.lang.Thread.run(Thread.java:833)
2024.02.08 15:17:10 INFO  web[][o.s.s.app.WebServer] Web Server stopped
2024.02.08 15:17:48 INFO  web[][o.s.p.ProcessEntryPoint] Starting Web Server
2024.02.08 15:17:49 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Starting...
2024.02.08 15:17:50 INFO  web[][c.z.h.p.HikariPool] HikariPool-1 - Added connection ConnectionID:1 ClientConnectionId: 30305d24-e4e3-445b-bb6a-69e36cd4fe89
2024.02.08 15:17:50 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Start completed.
2024.02.08 15:17:51 INFO  web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled
2024.02.08 15:17:51 INFO  web[][o.s.c.e.CoreExtensionsLoader] Loaded core extensions: enterprise-edition, ce-workers, developer-scanner, developer-server, governance, license, securityreport, monitoring, scim, gitlab-vulnerability-report
2024.02.08 15:17:52 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Clean as You Code / 2.1.0.500 / 4a2d47cf125d03ebacf43536a3897c168deb1b0a
2024.02.08 15:17:52 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Configuration detection for Code Quality and Security / 1.3.0.654 / 63073f0270b2c4754afa58eb8b5ea04e2eebf1a4
2024.02.08 15:17:52 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Dataflow Bug Detection / 1.17.0.4892 / 0
2024.02.08 15:17:56 INFO  web[][o.s.s.p.d.m.c.MssqlCharsetHandler] Verify that database collation is case-sensitive and accent-sensitive
2024.02.08 15:17:57 INFO  web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [http://127.0.0.1:9001]
2024.02.08 15:17:57 WARN  web[][o.s.a.s.w.WebService$Action] Description is not set on action api/monitoring/metrics
2024.02.08 15:17:57 WARN  web[][o.s.a.s.w.WebService$Action] Since is not set on action api/monitoring/metrics
2024.02.08 15:17:57 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/monitoring/metrics
2024.02.08 15:17:57 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/system/liveness
2024.02.08 15:17:57 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.platform.web.WebServiceFilter@283d3628 [pattern=UrlPattern{inclusions=[/api/system/migrate_db.*, ...], exclusions=[/api/components/update_key, ...]}]
2024.02.08 15:17:57 INFO  web[][o.s.s.p.DetectPluginChange] Detect plugin changes
2024.02.08 15:17:57 INFO  web[][o.s.s.p.DetectPluginChange] No plugin change detected
2024.02.08 15:17:59 INFO  web[][o.h.v.i.util.Version] HV000001: Hibernate Validator null
2024.02.08 15:18:00 INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2024.02.08 15:18:00 INFO  web[][o.s.s.s.LogServerId] Server ID: 34591E45-AYU-yIw6EWp9R7V5lpI6
2024.02.08 15:18:00 WARN  web[][o.s.s.a.LogOAuthWarning] For security reasons, OAuth authentication should use HTTPS. You should set the property 'Administration > Configuration > Server base URL' to a HTTPS URL.
2024.02.08 15:18:01 INFO  web[][o.s.s.p.UpdateCenterClient] Update center: https://update.sonarsource.org/update-center.properties
2024.02.08 15:18:02 INFO  web[][o.s.a.l.LdapSettingsManager] User mapping: LdapUserMapping{baseDn=DC=oddo,DC=fr, request=(&(objectCategory=Person)(sAMAccountName={0})), realNameAttribute=cn, emailAttribute=mail}
2024.02.08 15:18:02 INFO  web[][o.s.a.l.LdapSettingsManager] Group mapping: LdapGroupMapping{baseDn=OU=Application,OU=_Ressources,DC=oddo,DC=fr, idAttribute=sAMAccountName, requiredUserAttributes=[dn], request=(&(objectClass=group)(member={0}))}
2024.02.08 15:18:02 INFO  web[][o.s.a.l.LdapContextFactory] Test LDAP connection on ldap://******: OK
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action saml/validation_init
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/issues/list
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/system/liveness
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/plugins/download
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/analysis_cache/get
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/alm_integrations/check_pat
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/push/sonarlint_events
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/dismiss_message/check
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/security_reports/download
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/regulatory_reports/download
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/governance_reports/download
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/support/info
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/scim_management/status
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] Since is not set on action api/cayc/issues_creation_histogram
2024.02.08 15:18:02 WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/cayc/issues_creation_histogram
2024.02.08 15:18:03 INFO  web[][o.s.s.a.p.ExpiredSessionsCleaner] Purge of expired session tokens has removed 1 elements
2024.02.08 15:18:03 INFO  web[][o.s.s.a.p.ExpiredSessionsCleaner] Purge of expired SAML message ids has removed 0 elements
2024.02.08 15:18:03 INFO  web[][o.s.s.n.NotificationDaemon] Notification service started (delay 60 sec.)
2024.02.08 15:18:03 INFO  web[][o.s.s.t.TelemetryDaemon] Sharing of SonarQube statistics is enabled.
2024.02.08 15:18:03 INFO  web[][o.s.s.s.GeneratePluginIndex] Generate scanner plugin index
2024.02.08 15:18:03 INFO  web[][o.s.s.s.RegisterPermissionTemplates] Register permission templates
2024.02.08 15:18:03 INFO  web[][o.s.s.s.RenameDeprecatedPropertyKeys] Rename deprecated property keys
2024.02.08 15:18:03 INFO  web[][o.s.s.s.UpgradeSuggestionsCleaner] Dismissed messages cleanup
2024.02.08 15:18:03 INFO  web[][o.s.s.s.RegisterPlugins] Register plugins
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.platform.web.SonarLintConnectionFilter@f47bffe [pattern=UrlPattern{inclusions=[/api/*], exclusions=[/api/v2/*]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.platform.web.WebServiceFilter@3f8b440f [pattern=UrlPattern{inclusions=[/api/issues/delete_comment.*, ...], exclusions=[/api/authentication/login.*, ...]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.platform.web.WebServiceReroutingFilter@fff82a8 [pattern=UrlPattern{inclusions=[/api/components/update_key, ...], exclusions=[]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.DefaultAdminCredentialsVerifierFilter@4fc4b18e [pattern=UrlPattern{inclusions=[/*], exclusions=[*.css, ...]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.InitFilter@7d4e5fc7 [pattern=UrlPattern{inclusions=[/sessions/init/*], exclusions=[]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter com.A.B.B.A@34fc95d9 [pattern=UrlPattern{inclusions=[/api/scim/v2/Users, ...], exclusions=[]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter com.A.B.B.H@6edd58ed [pattern=UrlPattern{inclusions=[/api/scim/v2/Groups, ...], exclusions=[]}]
2024.02.08 15:18:03 INFO  web[][o.s.s.e.CoreExtensionBootstraper] Bootstrapping Governance
2024.02.08 15:18:03 INFO  web[][o.s.s.e.CoreExtensionBootstraper] Bootstrapping Governance (done) | time=0ms
2024.02.08 18:24:06 INFO  web[][o.s.s.n.NotificationDaemon] Notification service stopped
2024.02.08 18:24:06 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2024.02.08 18:24:06 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2024.02.08 18:24:06 WARN  web[][o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [pool-18-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:\n java.base@17.0.8/jdk.internal.misc.Unsafe.park(Native Method)\n java.base@17.0.8/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)\n java.base@17.0.8/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)\n java.base@17.0.8/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)\n java.base@17.0.8/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)\n java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)\n java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)\n java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n java.base@17.0.8/java.lang.Thread.run(Thread.java:833)
2024.02.08 18:24:06 WARN  web[][o.a.c.l.WebappClassLoaderBase] The web application [ROOT] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation. Stack trace of request processing thread:[\n java.base@17.0.8/jdk.internal.misc.Unsafe.park(Native Method)\n java.base@17.0.8/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)\n java.base@17.0.8/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:717)\n java.base@17.0.8/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:986)\n java.base@17.0.8/java.util.concurrent.locks.ReentrantLock$Sync.tryLockNanos(ReentrantLock.java:168)\n java.base@17.0.8/java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:479)\n java.naming@17.0.8/com.sun.jndi.ldap.pool.Connections.grabLock(Connections.java:363)\n java.naming@17.0.8/com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:140)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:341)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1608)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2848)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:348)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(LdapCtxFactory.java:225)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:189)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:152)\n java.naming@17.0.8/com.sun.jndi.url.ldap.ldapURLContextFactory.getObjectInstance(ldapURLContextFactory.java:52)\n java.naming@17.0.8/javax.naming.spi.NamingManager.getURLObject(NamingManager.java:625)\n java.naming@17.0.8/javax.naming.spi.NamingManager.processURL(NamingManager.java:402)\n java.naming@17.0.8/javax.naming.spi.NamingManager.processURLAddrs(NamingManager.java:382)\n java.naming@17.0.8/javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:354)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapReferralContext.<init>(LdapReferralContext.java:119)\n java.naming@17.0.8/com.sun.jndi.ldap.LdapReferralException.getReferralContext(LdapReferralException.java:151)\n java.naming@17.0.8/com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreReferrals(AbstractLdapNamingEnumeration.java:326)\n java.naming@17.0.8/com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreImpl(AbstractLdapNamingEnumeration.java:227)\n java.naming@17.0.8/com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreReferrals(AbstractLdapNamingEnumeration.java:348)\n java.naming@17.0.8/com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreImpl(AbstractLdapNamingEnumeration.java:227)\n java.naming@17.0.8/com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMore(AbstractLdapNamingEnumeration.java:189)\n app//org.sonar.auth.ldap.LdapSearch.hasMore(LdapSearch.java:156)\n app//org.sonar.auth.ldap.LdapSearch.findUnique(LdapSearch.java:146)\n app//org.sonar.auth.ldap.DefaultLdapGroupsProvider.searchUserGroups(DefaultLdapGroupsProvider.java:93)\n app//org.sonar.auth.ldap.DefaultLdapGroupsProvider.getGroups(DefaultLdapGroupsProvider.java:66)\n app//org.sonar.auth.ldap.DefaultLdapGroupsProvider.doGetGroups(DefaultLdapGroupsProvider.java:59)\n app//org.sonar.server.authentication.LdapCredentialsAuthentication.synchronize(LdapCredentialsAuthentication.java:128)\n app//org.sonar.server.authentication.LdapCredentialsAuthentication.doAuthenticate(LdapCredentialsAuthentication.java:100)\n app//org.sonar.server.authentication.LdapCredentialsAuthentication.authenticate(LdapCredentialsAuthentication.java:74)\n app//org.sonar.server.authentication.CredentialsAuthentication.lambda$authenticate$0(CredentialsAuthentication.java:71)\n app//org.sonar.server.authentication.CredentialsAuthentication$$Lambda$1584/0x000001e101b2d7e8.get(Unknown Source)\n java.base@17.0.8/java.util.Optional.or(Optional.java:313)\n app//org.sonar.server.authentication.CredentialsAuthentication.authenticate(CredentialsAuthentication.java:71)\n app//org.sonar.server.authentication.CredentialsAuthentication.authenticate(CredentialsAuthentication.java:57)\n app//org.sonar.server.authentication.BasicAuthentication.authenticate(BasicAuthentication.java:101)\n app//org.sonar.server.authentication.BasicAuthentication.lambda$authenticate$0(BasicAuthentication.java:54)\n app//org.sonar.server.authentication.BasicAuthentication$$Lambda$1583/0x000001e101b2d5a8.apply(Unknown Source)\n java.base@17.0.8/java.util.Optional.flatMap(Optional.java:289)\n app//org.sonar.server.authentication.BasicAuthentication.authenticate(BasicAuthentication.java:54)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl.lambda$loadUser$3(RequestAuthenticatorImpl.java:87)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl$$Lambda$1432/0x000001e101a83238.get(Unknown Source)\n java.base@17.0.8/java.util.Optional.or(Optional.java:313)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl.lambda$loadUser$4(RequestAuthenticatorImpl.java:87)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl$$Lambda$1421/0x000001e101a80a98.get(Unknown Source)\n java.base@17.0.8/java.util.Optional.orElseGet(Optional.java:364)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl.lambda$loadUser$5(RequestAuthenticatorImpl.java:85)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl$$Lambda$1418/0x000001e101a80000.get(Unknown Source)\n java.base@17.0.8/java.util.Optional.orElseGet(Optional.java:364)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl.loadUser(RequestAuthenticatorImpl.java:84)\n app//org.sonar.server.authentication.RequestAuthenticatorImpl.authenticate(RequestAuthenticatorImpl.java:68)\n app//org.sonar.server.authentication.UserSessionInitializer.loadUserSession(UserSessionInitializer.java:129)\n app//org.sonar.server.authentication.UserSessionInitializer.initUserSession(UserSessionInitializer.java:104)\n app//org.sonar.server.platform.web.UserSessionFilter.doFilter(UserSessionFilter.java:82)\n app//org.sonar.server.platform.web.UserSessionFilter.doFilter(UserSessionFilter.java:70)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\n app//org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:137)\n app//org.apache.catalina.core.ApplicationFilterChain$$Lambda$861/0x000001e101916cf8.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:569)\n app//org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:136)\n app//org.sonar.server.platform.web.CspFilter.doFilter(CspFilter.java:63)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\n app//org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:137)\n app//org.apache.catalina.core.ApplicationFilterChain$$Lambda$861/0x000001e101916cf8.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:569)\n app//org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:136)\n app//org.sonar.server.platform.web.CacheControlFilter.doFilter(CacheControlFilter.java:76)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\n app//org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:137)\n app//org.apache.catalina.core.ApplicationFilterChain$$Lambda$861/0x000001e101916cf8.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:569)\n app//org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:136)\n app//org.sonar.server.platform.web.SecurityServletFilter.doHttpFilter(SecurityServletFilter.java:60)\n app//org.sonar.server.platform.web.SecurityServletFilter.doFilter(SecurityServletFilter.java:47)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\n app//org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:137)\n app//org.apache.catalina.core.ApplicationFilterChain$$Lambda$861/0x000001e101916cf8.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:569)\n app//org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:136)\n app//org.sonar.server.platform.web.RedirectFilter.doFilter(RedirectFilter.java:56)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\n app//org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:137)\n app//org.apache.catalina.core.ApplicationFilterChain$$Lambda$861/0x000001e101916cf8.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:569)\n app//org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:136)\n app//org.sonar.server.platform.web.RequestIdFilter.doFilter(RequestIdFilter.java:66)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\n app//org.apache.catalina.core.ApplicationFilterChain.lambda$doFilter$0(ApplicationFilterChain.java:137)\n app//org.apache.catalina.core.ApplicationFilterChain$$Lambda$861/0x000001e101916cf8.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:569)\n app//org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:136)\n app//org.sonar.server.platform.web.RootFilter.doFilter(RootFilter.java:65)\n jdk.internal.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n java.base@17.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n java.base@17.0.8/java.lang.reflect.Method.invoke(Method.java:568)\n app//org.apache.catalina.security.SecurityUtil.lambda$execute$0(SecurityUtil.java:280)\n app//org.apache.catalina.security.SecurityUtil$$Lambda$682/0x000001e10156d988.run(Unknown Source)\n java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:807)\n java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:712)\n java.base@17.0.8/javax.security.auth.Subject.doAsPrivileged(Subject.java:584)\n app//org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:311)\n app//org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)\n app//org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:176)\

We checked the troubleshooting doc and I changed the maxQueryString and maxQueryStringLength and it seems to me that we have les instability. We’re receiving less socket timeout issues.

Also, we checked the hardware reccomandation for elasticssearch usage and we allocated 50% of the server’s memory to the search JVM

ganncamp · February 9, 2024, 3:12pm

Hi,

I’m not seeing any errors in the log snippet you’ve shared. Your 500-class response code indicates an error. Since you’re not finding any errors in your SonarQube log, this indicates the problem is elsewhere.

And specifically,

The 502 Bad Gateway error is an HTTP status code that occurs when a server acting as a gateway or proxy receives an invalid or faulty response from another server in the communication chain .

So again, you need to look outside SonarQube.

HTH,
Ann

Amine_Tanboura · March 22, 2024, 11:08am

Hello Ann,
We detected this error message after enabling the debug.
You can check also the log file
Log sonarqube 18_03.txt (987.0 KB)

Request http://sonarqube.oddo-bhf.com:9000/api/developers/search_events?projects=Clients-Aggregator&from=2024-03-14T16%3A50%3A37%2B0100 has been aborted by client
java.lang.IllegalStateException: Error while writing protobuf message

ganncamp · March 22, 2024, 4:06pm

Hi,

Here’s that error in more detail:

Request http://sonarqube.oddo-bhf.com:9000/api/developers/search_events?projects=Clients-Aggregator&from=2024-03-14T16%3A50%3A37%2B0100 has been aborted by client
java.lang.IllegalStateException: Error while writing protobuf message
...
Caused by: org.sonar.api.utils.text.WriterException: Fail to write JSON
...
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Connection reset by peer
...
Caused by: java.io.IOException: Connection reset by peer

aborted by client and Connection reset by peer indicate that the error logged here was caused externally to SonarQube. SonarQube was sending and the receiver just closed the connection.

I’m sorry, but this doesn’t change my answer: the problem is external to SonarQube.

Scrolling back in the history, I see a mention that you’re using SonarLint. Is that where you’re seeing the problem manifest? If so, I suggest making sure you’re on the latest SonarLint version.

Ann