SonarQube monitoring with Appdynamics

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • what are you trying to achieve
  • what have you tried so far to achieve this

I am running SQ version - 9.7.1.62043 and trying to configure JMX as mentioned here - Monitoring | SonarQube Docs
There are couple of issues I am having -

  1. SQ fails to start with authentication enabled because Elasticsearch is not coming up, there is not much information in sonar.log or es.log, I only see that ES is trying to start and then it eventually fails, the whole process goes in loops.

  2. After a bit of tweaking I figured out that SQ is starting only when I disable authentication, I am using these options
    sonar.web.javaAdditionalOpts=-Dcom.sun.management.jmxremote
    -Dcom.sun.management.jmxremote.port=10443
    -Dcom.sun.management.jmxremote.rmi.port=10444
    -Dcom.sun.management.jmxremote.authenticate=false
    -Dcom.sun.management.jmxremote.ssl=false

But when I add AppD java agent like below SQ does not start at all, I only see that ES is trying to start and then it eventually fails, the whole process goes in loops.

sonar.web.javaAdditionalOpts=-javaagent:/opt/appdynamics/java-agent/javaagent.jar
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=10443
-Dcom.sun.management.jmxremote.rmi.port=10444
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

What failure are you getting from Elasticsearch?

Hi Colin,

ES tries to start and then fails, the whole process repeats itself. I tried to delete the es7 folder but it still does not work, I am attaching sonar.log and es.log during the service start up process.

==> sonar.log <==
2022.11.24 14:53:11 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /apps/sq-shared/temp
2022.11.24 14:53:12 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:44553]
2022.11.24 14:53:12 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/apps/sonarqube-9.7.1.62043/elasticsearch]: /apps/sonarqube-9.7.1.62043/elasticsearch/bin/elasticsearch
2022.11.24 14:53:12 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running

==> es.log <==
2022.11.24 14:53:15 INFO  es[][o.e.n.Node] version[7.17.5], pid[32718], build[default/tar/8d61b4f7ddf931f219e3745f295ed2bbc50c8e84/2022-06-23T21:57:28.736740635Z], OS[Linux/4.14.294-220.533.amzn2.x86_64/amd64], JVM[Red Hat, Inc./OpenJDK 64-Bit Server VM/11.0.16/11.0.16+8-LTS]
2022.11.24 14:53:15 INFO  es[][o.e.n.Node] JVM home [/usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.amzn2.0.1.x86_64]
2022.11.24 14:53:15 INFO  es[][o.e.n.Node] JVM arguments [-XX:+UseG1GC, -Djava.io.tmpdir=/apps/sq-shared/temp, -XX:ErrorFile=../logs/es_hs_err_pid%p.log, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/apps/sq-shared/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Des.enforce.bootstrap.checks=true, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.policy=/apps/sonarqube_latest/elasticsearch/config/security.policy, -Des.path.home=/apps/sonarqube-9.7.1.62043/elasticsearch, -Des.path.conf=/apps/sq-shared/temp/conf/es, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=false]
2022.11.24 14:53:16 INFO  es[][o.e.p.PluginsService] loaded module [analysis-common]
2022.11.24 14:53:16 INFO  es[][o.e.p.PluginsService] loaded module [lang-painless]
2022.11.24 14:53:16 INFO  es[][o.e.p.PluginsService] loaded module [parent-join]
2022.11.24 14:53:16 INFO  es[][o.e.p.PluginsService] loaded module [reindex]
2022.11.24 14:53:16 INFO  es[][o.e.p.PluginsService] loaded module [transport-netty4]
2022.11.24 14:53:16 INFO  es[][o.e.p.PluginsService] no plugins loaded
2022.11.24 14:53:16 INFO  es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/apps/sq-shared (myorgefs.efs.us-west-2.amazonaws.com:/)]], net usable_space [8191.9pb], net total_space [8191.9pb], types [nfs4]
2022.11.24 14:53:16 INFO  es[][o.e.e.NodeEnvironment] heap size [512mb], compressed ordinary object pointers [true]
2022.11.24 14:53:18 INFO  es[][o.e.n.Node] node name [sonarqube], node ID [NO9_ffxPRgSTOSedbQOYOw], cluster name [sonarqube], roles [data_frozen, master, remote_cluster_client, data, data_content, data_hot, data_warm, data_cold, ingest]
2022.11.24 14:53:22 INFO  es[][o.e.t.NettyAllocator] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=512mb}]
2022.11.24 14:53:22 INFO  es[][o.e.i.r.RecoverySettings] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
2022.11.24 14:53:22 INFO  es[][o.e.d.DiscoveryModule] using discovery type [zen] and seed hosts providers [settings]
2022.11.24 14:53:22 INFO  es[][o.e.g.DanglingIndicesState] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
2022.11.24 14:53:23 INFO  es[][o.e.n.Node] initialized
2022.11.24 14:53:23 INFO  es[][o.e.n.Node] starting ...
2022.11.24 14:53:23 INFO  es[][o.e.t.TransportService] publish_address {127.0.0.1:44553}, bound_addresses {127.0.0.1:44553}
2022.11.24 14:53:24 INFO  es[][o.e.b.BootstrapChecks] explicitly enforcing bootstrap checks
2022.11.24 14:53:24 INFO  es[][o.e.c.c.Coordinator] cluster UUID [v-VG7SNbTASOJHzZSbiC8A]
2022.11.24 14:53:24 INFO  es[][o.e.c.s.MasterService] elected-as-master ([1] nodes joined)[{sonarqube}{NO9_ffxPRgSTOSedbQOYOw}{C14mYgBvQDioQfVZ_b5oSQ}{127.0.0.1}{127.0.0.1:44553}{cdfhimrsw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 973, version: 19415, delta: master node changed {previous [], current [{sonarqube}{NO9_ffxPRgSTOSedbQOYOw}{C14mYgBvQDioQfVZ_b5oSQ}{127.0.0.1}{127.0.0.1:44553}{cdfhimrsw}]}
2022.11.24 14:53:24 INFO  es[][o.e.c.s.ClusterApplierService] master node changed {previous [], current [{sonarqube}{NO9_ffxPRgSTOSedbQOYOw}{C14mYgBvQDioQfVZ_b5oSQ}{127.0.0.1}{127.0.0.1:44553}{cdfhimrsw}]}, term: 973, version: 19415, reason: Publication{term=973, version=19415}
2022.11.24 14:53:24 INFO  es[][o.e.h.AbstractHttpServerTransport] publish_address {127.0.0.1:9001}, bound_addresses {127.0.0.1:9001}
2022.11.24 14:53:24 INFO  es[][o.e.n.Node] started
2022.11.24 14:53:24 INFO  es[][o.e.g.GatewayService] recovered [7] indices into cluster_state
2022.11.24 14:53:31 INFO  es[][o.e.c.r.a.AllocationService] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[metadatas][0]]]).

==> sonar.log <==
2022.11.24 14:53:31 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
2022.11.24 14:53:31 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[WEB_SERVER] from [/apps/sonarqube-9.7.1.62043]: /usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.amzn2.0.1.x86_64/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/apps/sq-shared/temp -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -Dcom.redhat.fips=false -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -javaagent:/opt/appdynamics/java-agent/javaagent.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10443 -Dcom.sun.management.jmxremote.rmi.port=10444 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dhttp.proxyHost=myproxy.dcmyorg.com -Dhttp.proxyPort=1234 -Dhttp.nonProxyHosts=localhost|127.*|[::1] -Dhttps.proxyHost=myproxy.dcmyorg.com -Dhttps.proxyPort=1234 -cp ./lib/sonar-application-9.7.1.62043.jar:/apps/sonarqube-9.7.1.62043/lib/jdbc/postgresql/postgresql-42.4.1.jar org.sonar.server.app.WebServer /apps/sq-shared/temp/sq-process9625845962002219812properties
2022.11.24 14:54:09 INFO  app[][o.s.a.SchedulerImpl] Process[Web Server] is stopped

==> es.log <==
2022.11.24 14:54:09 INFO  es[][o.e.n.Node] stopping ...
2022.11.24 14:54:09 INFO  es[][o.e.n.Node] stopped
2022.11.24 14:54:09 INFO  es[][o.e.n.Node] closing ...
2022.11.24 14:54:09 INFO  es[][o.e.n.Node] closed

==> sonar.log <==
2022.11.24 14:54:09 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 143
2022.11.24 14:54:09 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2022.11.24 14:54:09 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
2022.11.24 14:54:10 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /apps/sq-shared/temp
2022.11.24 14:54:10 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:37905]
2022.11.24 14:54:11 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/apps/sonarqube-9.7.1.62043/elasticsearch]: /apps/sonarqube-9.7.1.62043/elasticsearch/bin/elasticsearch
2022.11.24 14:54:11 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running

==> es.log <==
2022.11.24 14:54:13 INFO  es[][o.e.n.Node] version[7.17.5], pid[6092], build[default/tar/8d61b4f7ddf931f219e3745f295ed2bbc50c8e84/2022-06-23T21:57:28.736740635Z], OS[Linux/4.14.294-220.533.amzn2.x86_64/amd64], JVM[Red Hat, Inc./OpenJDK 64-Bit Server VM/11.0.16/11.0.16+8-LTS]
2022.11.24 14:54:13 INFO  es[][o.e.n.Node] JVM home [/usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.amzn2.0.1.x86_64]
2022.11.24 14:54:13 INFO  es[][o.e.n.Node] JVM arguments [-XX:+UseG1GC, -Djava.io.tmpdir=/apps/sq-shared/temp, -XX:ErrorFile=../logs/es_hs_err_pid%p.log, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/apps/sq-shared/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Des.enforce.bootstrap.checks=true, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.policy=/apps/sonarqube_latest/elasticsearch/config/security.policy, -Des.path.home=/apps/sonarqube-9.7.1.62043/elasticsearch, -Des.path.conf=/apps/sq-shared/temp/conf/es, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=false]
2022.11.24 14:54:14 INFO  es[][o.e.p.PluginsService] loaded module [analysis-common]
2022.11.24 14:54:14 INFO  es[][o.e.p.PluginsService] loaded module [lang-painless]
2022.11.24 14:54:14 INFO  es[][o.e.p.PluginsService] loaded module [parent-join]
2022.11.24 14:54:14 INFO  es[][o.e.p.PluginsService] loaded module [reindex]
2022.11.24 14:54:14 INFO  es[][o.e.p.PluginsService] loaded module [transport-netty4]
2022.11.24 14:54:14 INFO  es[][o.e.p.PluginsService] no plugins loaded
2022.11.24 14:54:14 INFO  es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/apps/sq-shared (myorgefs.efs.us-west-2.amazonaws.com:/)]], net usable_space [8191.9pb], net total_space [8191.9pb], types [nfs4]
2022.11.24 14:54:14 INFO  es[][o.e.e.NodeEnvironment] heap size [512mb], compressed ordinary object pointers [true]
2022.11.24 14:54:15 INFO  es[][o.e.n.Node] node name [sonarqube], node ID [NO9_ffxPRgSTOSedbQOYOw], cluster name [sonarqube], roles [data_frozen, master, remote_cluster_client, data, data_content, data_hot, data_warm, data_cold, ingest]
2022.11.24 14:54:19 INFO  es[][o.e.t.NettyAllocator] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=512mb}]
2022.11.24 14:54:19 INFO  es[][o.e.i.r.RecoverySettings] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
2022.11.24 14:54:19 INFO  es[][o.e.d.DiscoveryModule] using discovery type [zen] and seed hosts providers [settings]
2022.11.24 14:54:20 INFO  es[][o.e.g.DanglingIndicesState] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
2022.11.24 14:54:20 INFO  es[][o.e.n.Node] initialized
2022.11.24 14:54:20 INFO  es[][o.e.n.Node] starting ...
2022.11.24 14:54:20 INFO  es[][o.e.t.TransportService] publish_address {127.0.0.1:37905}, bound_addresses {127.0.0.1:37905}
2022.11.24 14:54:21 INFO  es[][o.e.b.BootstrapChecks] explicitly enforcing bootstrap checks
2022.11.24 14:54:21 INFO  es[][o.e.c.c.Coordinator] cluster UUID [v-VG7SNbTASOJHzZSbiC8A]
2022.11.24 14:54:21 INFO  es[][o.e.c.s.MasterService] elected-as-master ([1] nodes joined)[{sonarqube}{NO9_ffxPRgSTOSedbQOYOw}{JKUf8taSSOu-7GmMw4E6wA}{127.0.0.1}{127.0.0.1:37905}{cdfhimrsw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 974, version: 19437, delta: master node changed {previous [], current [{sonarqube}{NO9_ffxPRgSTOSedbQOYOw}{JKUf8taSSOu-7GmMw4E6wA}{127.0.0.1}{127.0.0.1:37905}{cdfhimrsw}]}
2022.11.24 14:54:22 INFO  es[][o.e.c.s.ClusterApplierService] master node changed {previous [], current [{sonarqube}{NO9_ffxPRgSTOSedbQOYOw}{JKUf8taSSOu-7GmMw4E6wA}{127.0.0.1}{127.0.0.1:37905}{cdfhimrsw}]}, term: 974, version: 19437, reason: Publication{term=974, version=19437}
2022.11.24 14:54:22 INFO  es[][o.e.h.AbstractHttpServerTransport] publish_address {127.0.0.1:9001}, bound_addresses {127.0.0.1:9001}
2022.11.24 14:54:22 INFO  es[][o.e.n.Node] started
2022.11.24 14:54:22 INFO  es[][o.e.g.GatewayService] recovered [7] indices into cluster_state
2022.11.24 14:54:28 INFO  es[][o.e.c.r.a.AllocationService] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[components][3], [components][4], [metadatas][0]]]).

==> sonar.log <==
2022.11.24 14:54:28 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
2022.11.24 14:54:28 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[WEB_SERVER] from [/apps/sonarqube-9.7.1.62043]: /usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.amzn2.0.1.x86_64/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/apps/sq-shared/temp -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -Dcom.redhat.fips=false -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -javaagent:/opt/appdynamics/java-agent/javaagent.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10443 -Dcom.sun.management.jmxremote.rmi.port=10444 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dhttp.proxyHost=myproxy.dcmyorg.com -Dhttp.proxyPort=1234 -Dhttp.nonProxyHosts=localhost|127.*|[::1] -Dhttps.proxyHost=myproxy.dcmyorg.com -Dhttps.proxyPort=1234 -cp ./lib/sonar-application-9.7.1.62043.jar:/apps/sonarqube-9.7.1.62043/lib/jdbc/postgresql/postgresql-42.4.1.jar org.sonar.server.app.WebServer /apps/sq-shared/temp/sq-process8808377357892672642properties
2022.11.24 14:55:02 INFO  app[][o.s.a.SchedulerImpl] Process[Web Server] is stopped

==> es.log <==
2022.11.24 14:55:02 INFO  es[][o.e.n.Node] stopping ...
2022.11.24 14:55:02 INFO  es[][o.e.n.Node] stopped
2022.11.24 14:55:02 INFO  es[][o.e.n.Node] closing ...
2022.11.24 14:55:02 INFO  es[][o.e.n.Node] closed

==> sonar.log <==
2022.11.24 14:55:02 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 143
2022.11.24 14:55:02 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2022.11.24 14:55:02 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
2022.11.24 14:55:03 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /apps/sq-shared/temp
2022.11.24 14:55:03 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:40435]
2022.11.24 14:55:03 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/apps/sonarqube-9.7.1.62043/elasticsearch]: /apps/sonarqube-9.7.1.62043/elasticsearch/bin/elasticsearch
2022.11.24 14:55:03 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running

This tells me that the real issue will be found in the web.log file. Is there anything there?

I did a tail -f *.log in the logs directory, anything came up for web.log

I enabled debug logs and captured the web logs, I see some errors related to AppD

2022.11.24 16:06:29 INFO  web[][o.s.p.ProcessEntryPoint] Starting Web Server
2022.11.24 16:06:32 INFO  web[][o.s.s.p.LogServerVersion] SonarQube Server / 9.7.1.62043 / 13c69ab992096c497badccbdb4a31110594574fd
2022.11.24 16:06:32 INFO  web[][o.sonar.db.Database] Create JDBC data source for jdbc:postgresql://sonarqube-db-cluster.us-west-2.rds.amazonaws.com/sonarqube
2022.11.24 16:06:32 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Starting...
2022.11.24 16:06:32 INFO  web[][c.z.h.p.HikariPool] HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@4d4436d0
2022.11.24 16:06:32 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Start completed.
2022.11.24 16:06:34 INFO  web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /apps/sonarqube-9.7.1.62043
2022.11.24 16:06:34 INFO  web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled
2022.11.24 16:06:35 INFO  web[][o.s.s.p.ServerPluginManager] Deploy C# Code Quality and Security / 8.46.0.54807 / a9f10bf0c980dd16d428c08c7a984e2205a512df
2022.11.24 16:06:35 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Python Code Quality and Security / 3.19.0.10254 / 411866233dc7e1f384592447884d27e220cb00ec
2022.11.24 16:06:35 INFO  web[][o.s.s.p.ServerPluginManager] Deploy YAML Analyzer / 1.7.0 / null
2022.11.24 16:06:35 INFO  web[][o.s.s.p.ServerPluginManager] Deploy miniOrange SonarQube SAML SSO Plugin / 1.1.3 / null
2022.11.24 16:06:40 WARN  web[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@49476842-org.sonar.server.plugins.ServerPluginManager': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to instantiate class [com.github.sbaudoin.sonar.plugins.yaml.YamlPlugin] of plugin [yaml]
2022.11.24 16:06:40 ERROR web[][o.s.s.p.PlatformImpl] Web server startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@49476842-org.sonar.server.plugins.ServerPluginManager': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to instantiate class [com.github.sbaudoin.sonar.plugins.yaml.YamlPlugin] of plugin [yaml]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:187)
	at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
	at org.sonar.server.platform.platformlevel.PlatformLevel2.start(PlatformLevel2.java:105)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:196)
	at org.sonar.server.platform.PlatformImpl.startLevel2Container(PlatformImpl.java:169)
	at org.sonar.server.platform.PlatformImpl.init(PlatformImpl.java:77)
	at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:43)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4768)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5230)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:919)
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:919)
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:263)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:432)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:927)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.startup.Tomcat.start(Tomcat.java:486)
	at org.sonar.server.app.EmbeddedTomcat.start(EmbeddedTomcat.java:72)
	at org.sonar.server.app.WebServer.start(WebServer.java:55)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:97)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:81)
	at org.sonar.server.app.WebServer.main(WebServer.java:104)
Caused by: java.lang.IllegalStateException: Fail to instantiate class [com.github.sbaudoin.sonar.plugins.yaml.YamlPlugin] of plugin [yaml]
	at org.sonar.core.platform.PluginClassLoader.instantiatePluginClasses(PluginClassLoader.java:135)
	at org.sonar.core.platform.PluginClassLoader.load(PluginClassLoader.java:67)
	at org.sonar.core.platform.PluginClassLoader.load(PluginClassLoader.java:61)
	at org.sonar.server.plugins.ServerPluginManager.start(ServerPluginManager.java:68)
	at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	... 44 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.sonar.core.platform.PluginClassLoader.instantiatePluginClasses(PluginClassLoader.java:131)
	... 51 common frames omitted
Caused by: java.lang.StackOverflowError: null
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.logging/java.util.logging.LogManager.checkPermission(LogManager.java:2432)
	at java.logging/java.util.logging.Logger.checkPermission(Logger.java:622)
	at java.logging/java.util.logging.Logger.setLevel(Logger.java:2001)
	at ch.qos.logback.classic.jul.LevelChangePropagator.propagate(LevelChangePropagator.java:66)
	at ch.qos.logback.classic.jul.LevelChangePropagator.onLevelChange(LevelChangePropagator.java:56)
	at ch.qos.logback.classic.LoggerContext.fireOnLevelChange(LoggerContext.java:318)
	at ch.qos.logback.classic.Logger.setLevel(Logger.java:173)
	at com.github.sbaudoin.sonar.plugins.yaml.YamlPlugin.<init>(YamlPlugin.java:38)
	... 56 common frames omitted
2022.11.24 16:06:40 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2022.11.24 16:06:40 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2022.11.24 16:06:40 WARN  web[][o.a.c.l.WebappClassLoaderBase] Failed to clear soft references from ObjectStreamClass$Caches for web application [ROOT]
java.lang.ClassCastException: class java.io.ObjectStreamClass$Caches$1 cannot be cast to class java.util.Map (java.io.ObjectStreamClass$Caches$1 and java.util.Map are in module java.base of loader 'bootstrap')
	at org.apache.catalina.loader.WebappClassLoaderBase.clearCache(WebappClassLoaderBase.java:2336)
	at org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesObjectStreamClassCaches(WebappClassLoaderBase.java:2311)
	at org.apache.catalina.loader.WebappClassLoaderBase.clearReferences(WebappClassLoaderBase.java:1674)
	at org.apache.catalina.loader.WebappClassLoaderBase.stop(WebappClassLoaderBase.java:1602)
	at org.apache.catalina.loader.WebappLoader.stopInternal(WebappLoader.java:463)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5515)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:187)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:919)
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:919)
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:263)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:432)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:927)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.startup.Tomcat.start(Tomcat.java:486)
	at org.sonar.server.app.EmbeddedTomcat.start(EmbeddedTomcat.java:72)
	at org.sonar.server.app.WebServer.start(WebServer.java:55)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:97)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:81)
	at org.sonar.server.app.WebServer.main(WebServer.java:104)
2022.11.24 16:06:41 INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2022.11.24 16:06:41 INFO  web[][o.s.p.ProcessEntryPoint] Hard stopping process
2022.11.24 16:06:41 WARN  web[][o.a.c.l.WebappClassLoaderBase] Failed to clear soft references from ObjectStreamClass$Caches for web application [deploy]
java.lang.ClassCastException: class java.io.ObjectStreamClass$Caches$1 cannot be cast to class java.util.Map (java.io.ObjectStreamClass$Caches$1 and java.util.Map are in module java.base of loader 'bootstrap')
	at org.apache.catalina.loader.WebappClassLoaderBase.clearCache(WebappClassLoaderBase.java:2336)
	at org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesObjectStreamClassCaches(WebappClassLoaderBase.java:2311)
	at org.apache.catalina.loader.WebappClassLoaderBase.clearReferences(WebappClassLoaderBase.java:1674)
	at org.apache.catalina.loader.WebappClassLoaderBase.stop(WebappClassLoaderBase.java:1602)
	at org.apache.catalina.loader.WebappLoader.stopInternal(WebappLoader.java:463)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5515)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1412)
	at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1401)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.stopInternal(ContainerBase.java:986)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1412)
	at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1401)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.stopInternal(ContainerBase.java:986)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.core.StandardService.stopInternal(StandardService.java:497)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.core.StandardServer.stopInternal(StandardServer.java:979)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:257)
	at org.apache.catalina.startup.Tomcat.stop(Tomcat.java:496)
	at org.sonar.server.app.EmbeddedTomcat.terminate(EmbeddedTomcat.java:112)
	at org.sonar.server.app.WebServer.hardStop(WebServer.java:86)
	at org.sonar.process.ProcessEntryPoint$HardStopperThread.lambda$new$0(ProcessEntryPoint.java:219)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
2022.11.24 16:07:22 INFO  web[][o.s.p.ProcessEntryPoint] Starting Web Server

It looks like the root cause is something to do with this community-supported plugin:

You might want to raise an issue on their GitHub repository – and in the meantime, try uninstalling it to see if it otherwise allows SonarQube to startup.

Hi Colin,

I un-installed the Yaml plugin and also for testing I removed the default plugins form /sonar-home/lib/extensions/ directory, I still see some errors.

2022.11.25 15:19:05 WARN  web[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@49476842-org.sonar.server.ui.PageRepository': Initialization of bean failed; nested exception is java.lang.StackOverflowError
2022.11.25 15:19:05 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@49476842-org.sonar.server.ui.PageRepository': Initialization of bean failed; nested exception is java.lang.StackOverflowError
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:187)
	at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
	at org.sonar.server.platform.platformlevel.PlatformLevel4.start(PlatformLevel4.java:647)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:196)
	at org.sonar.server.platform.PlatformImpl.startLevel34Containers(PlatformImpl.java:177)
	at org.sonar.server.platform.PlatformImpl$1.lambda$doRun$0(PlatformImpl.java:105)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.runIfNotAborted(PlatformImpl.java:344)
	at org.sonar.server.platform.PlatformImpl$1.doRun(PlatformImpl.java:105)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.run(PlatformImpl.java:328)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.StackOverflowError: null
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)

I did a bit of tweaking with the JVM arguments and added this sonar.web.javaOpts=-Xmx1G -Xms1G -Xss12m -XX:+HeapDumpOnOutOfMemoryError to the sonar.properties file. It helped me get passed the initial error but it failed to start the default plugins and gave errors similar to below:

java.lang.IllegalStateException: Fail to load plugin C# Code Quality and Security [csharp]
	at org.sonar.server.plugins.ServerExtensionInstaller.installExtensions(ServerExtensionInstaller.java:81)
	at org.sonar.server.platform.platformlevel.PlatformLevel4.start(PlatformLevel4.java:645)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:196)
	at org.sonar.server.platform.PlatformImpl.startLevel34Containers(PlatformImpl.java:177)
	at org.sonar.server.platform.PlatformImpl$1.lambda$doRun$0(PlatformImpl.java:105)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.runIfNotAborted(PlatformImpl.java:344)
	at org.sonar.server.platform.PlatformImpl$1.doRun(PlatformImpl.java:105)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.run(PlatformImpl.java:328)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.StackOverflowError: null
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)
	at com.singularity.ee.agent.appagent.entrypoint.bciengine.AnonymousClassDefTransformerBoot.classDefTrap(AnonymousClassDefTransformerBoot.java:31)
	at java.base/jdk.internal.misc.Unsafe.defineAnonymousClass(Unsafe.java:1223)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:331)
	at java.base/java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:195)
	at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:329)
	at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127)
	at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
	at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
	at org.sonar.process.PluginSecurityManager$PluginPolicy.implies(PluginSecurityManager.java:61)
	at java.base/java.security.ProtectionDomain.implies(ProtectionDomain.java:321)
	at java.base/java.security.ProtectionDomain.impliesWithAltFilePerm(ProtectionDomain.java:353)
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2060)
	at java.base/java.lang.Class.getClassLoader(Class.java:830)
	at com.appdynamics.appagent/com.singularity.ee.agent.appagent.services.bciengine.transformation.AnonymousClassDefTransformer.classDefTrap(AnonymousClassDefTransformer.java:61)

I removed the C# plugin but I got error for other plugins which are part of the lib/extensions/plugins folder. I went ahead a removed all the plugins after which I was able to start the application but without the default plugins for various languages like Java, c# etc is of no use.

Thanks for the update.

It looks like Java’s own security manager (which was turned on with [SONAR-14886] - Jira) is getting in the way. We do this for security reasons – I’m not sure if there’s still a way to attach a java agent (legitimately) to a process. I’ve flagged this for some expert attention.

Do we have any further updates on this ?

Hello @Kashif,

For your initial problem about authentication preventing SonarQube to set, I think the reason is simply that you didn’t provide password/access files:

-Dcom.sun.management.jmxremote.password.file=/<sqlocation>/jmxremote.password 
-Dcom.sun.management.jmxremote.access.file=/<sqlocation>/jmxremote.access 

With those 2 properties set, and the files filed with correct information and permissions (see https://docs.oracle.com/javadb/10.10.1.2/adminguide/radminjmxenablepwd.html, https://docs.oracle.com/javadb/10.10.1.2/adminguide/radminjmxenablesimpleauth.html), SonarQube starts correctly.

After that, I’ve tried to add the appdynamics agent, version 22.11.0, and it still works all fine. Here is the full set of java options I use:

-Dcom.sun.management.jmxremote=true 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=true 
-Dcom.sun.management.jmxremote.port=10443 
-Dcom.sun.management.jmxremote.rmi.port=10444 
-Dcom.sun.management.jmxremote.password.file=/<sqLocation>/sonar-enterprise/jmxremote.password 
-Dcom.sun.management.jmxremote.access.file=/<sqLocation>/sonar-enterprise/jmxremote.access 
-javaagent:/<sqLocation>/javaagent.jar

Can you retry making sure you have all those arguments?

If still not working, can you tell me on which OS you are? can you try to launch another java process with the same options to see if the agent correctly starts?

we are using sonarqube version 20.8.0.30686 and Amazon Linux release 2.

i updated my config to below

sonar.web.javaAdditionalOpts=-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.port=10443 -Dcom.sun.management.jmxremote.rmi.port=10444 -Dcom.sun.management.jmxremote.password.file=/apps/sonarqube_latest/conf/jmxremote.password -Dcom.sun.management.jmxremote.access.file=/apps/sonarqube_latest/conf/jmxremote.access -javaagent:/opt/appdynamics/java-agent/javaagent.jar

but when starting the service,still webserver gets stopped though java agent logs tells it had started the process

Web server logs :

2023.01.16 14:33:21 INFO  web[][o.s.p.ProcessEntryPoint] Gracefully stopping process
2023.01.16 14:33:21 INFO  web[][o.s.s.n.NotificationDaemon] Notification service stopped
2023.01.16 14:33:21 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2023.01.16 14:33:21 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2023.01.16 14:33:22 INFO  web[][o.s.s.app.WebServer] Web Server stopped

Java Agent logs :

/opt/appdynamics/java-agent/ver20.8.0.30686/logs/Auto/BusinessTransactions.2023_01_16__14_51_47.log <==
[AD Agent init] 16 Jan 2023 14:51:48,018  INFO - Starting BT Logs at Mon Jan 16 14:51:48 UTC 2023
[AD Agent init] 16 Jan 2023 14:51:48,023  INFO - ###########################################################
[AD Agent init] 16 Jan 2023 14:51:48,023  INFO - Using Java Agent Version [Server Agent #20.8.0.30686 v20.8.0 GA compatible with 4.4.1.0 rcaedb006dc46dddce17bc7540c4bed10fa52536e master]
[AD Agent init] 16 Jan 2023 14:51:48,023  INFO - Running IBM Java Agent [No]

```

Hi @Karthik_T,

This is not a SonarQube version. The current maximal version is 9.8. Can you double-check this number?

Can you send me the full web logs, so that I can see what happens before SonarQube gets shutdown?