[SonarQube 7.7] Does not start on CentOS Linux

Title says everything. Same error happens with SonarQube 6.7.7 LTS but I write only 7.7 issues here. I have no version restriction which I should use.

I downloaded SonarQube 7.7 from your website. Configured limits as following:

# cat /etc/security/limits.conf | grep sonar
sonarqube   -   nofile   65536
sonarqube   -   nproc    4096

In sysctl.conf I set proper values for things:

vm.max_map_count=262144
fs.file-max=65536

I also applied these values after configuring.

After unpacking the zip to /opt/sonarqube/sonarqube-7.7 I chowned the whole directory to the newly created user sonarqube:

chown -R sonarqube:sonarqube /opt/sonarqube

After it, I configured sonar.properties as following (removed comments from the config for better readability:

sonar.jdbc.username=sonarq
sonar.jdbc.url=jdbc:postgresql://localhost/sonarq_production
http.proxyHost=aproxy.some.where
http.proxyUser=proxyuser
http.proxyPassword=proxypass

PostgreSQL has the specified SQL user and database and triple checked it could login with these details.

After trying to start sonarqube with

sudo -Hu sonarqube ./bin/linux-x86-64/sonar.sh console 

I got the following output

Running SonarQube...
wrapper  | --> Wrapper Started as Console
wrapper  | Launching a JVM...
jvm 1    | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
jvm 1    |   Copyright 1999-2006 Tanuki Software, Inc.  All Rights Reserved.
jvm 1    |
jvm 1    | 2019.06.12 16:51:43 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/sonarqube-7.7/temp
jvm 1    | 2019.06.12 16:51:43 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
jvm 1    | 2019.06.12 16:51:43 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/sonarqube-7.7/elasticsearch]: /opt/sonarqube/sonarqube-7.7/elasticsearch/bin/elasticsearch
jvm 1    | 2019.06.12 16:51:44 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm 1    | 2019.06.12 16:51:44 INFO  app[][o.e.p.PluginsService] no modules loaded
jvm 1    | 2019.06.12 16:51:44 INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm 1    | 2019.06.12 16:51:58 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
jvm 1    | 2019.06.12 16:51:58 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=2, logFilenamePrefix=web]] from [/opt/sonarqube/sonarqube-7.7]: /usr/java/jdk1.8.0_212-amd64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/sonarqube-7.7/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttp.proxyHost=aproxy.some.where -Dhttps.proxyHost=aproxy.some.where -cp ./lib/common/*:/opt/sonarqube/sonarqube-7.7/lib/jdbc/postgresql/postgresql-42.2.5.jar org.sonar.server.app.WebServer /opt/sonarqube/sonarqube-7.7/temp/sq-process5107517876487134023properties
jvm 1    | 2019.06.12 16:52:03 INFO  app[][o.s.a.SchedulerImpl] Process [web] is stopped
jvm 1    | 2019.06.12 16:52:03 INFO  app[][o.s.a.SchedulerImpl] Process [es] is stopped
jvm 1    | 2019.06.12 16:52:03 WARN  app[][o.s.a.p.AbstractProcessMonitor] Process exited with exit value [es]: 143
jvm 1    | 2019.06.12 16:52:03 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
wrapper  | <-- Wrapper Stopped

es.log says

2019.06.12 16:51:47 INFO  es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [44gb], net total_space [50.5gb], types [rootfs]
2019.06.12 16:51:47 INFO  es[][o.e.e.NodeEnvironment] heap size [495.3mb], compressed ordinary object pointers [true]
2019.06.12 16:51:47 INFO  es[][o.e.n.Node] node name [sonarqube], node ID [qT3jesniTNOvQuFTc150Lg]
2019.06.12 16:51:47 INFO  es[][o.e.n.Node] version[6.6.2], pid[10153], build[default/tar/3bd3e59/2019-03-06T15:16:26.864148Z], OS[Linux/3.10.0-957.5.1.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_212/25.212-b10]
2019.06.12 16:51:47 INFO  es[][o.e.n.Node] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/opt/sonarqube/sonarqube-7.7/temp/es6, -XX:ErrorFile=/opt/sonarqube/sonarqube-7.7/logs/es_hs_err_pid%p.log, -Xms512m, -Xmx512m, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/opt/sonarqube/sonarqube-7.7/elasticsearch, -Des.path.conf=/opt/sonarqube/sonarqube-7.7/temp/conf/es, -Des.distribution.flavor=default, -Des.distribution.type=tar]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [analysis-common]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [lang-painless]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [mapper-extras]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [parent-join]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [percolator]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [reindex]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [repository-url]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] loaded module [transport-netty4]
2019.06.12 16:51:48 INFO  es[][o.e.p.PluginsService] no plugins loaded
2019.06.12 16:51:52 WARN  es[][o.e.d.c.s.Settings] [http.enabled] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.
2019.06.12 16:51:53 INFO  es[][o.e.d.DiscoveryModule] using discovery type [zen] and host providers [settings]
2019.06.12 16:51:54 INFO  es[][o.e.n.Node] initialized
2019.06.12 16:51:54 INFO  es[][o.e.n.Node] starting ...
2019.06.12 16:51:54 INFO  es[][o.e.t.TransportService] publish_address {127.0.0.1:9001}, bound_addresses {127.0.0.1:9001}
2019.06.12 16:51:57 INFO  es[][o.e.c.s.MasterService] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {sonarqube}{qT3jesniTNOvQuFTc150Lg}{5wBh0CdCRxqzg0TXK1029Q}{127.0.0.1}{127.0.0.1:9001}{rack_id=sonarqube}
2019.06.12 16:51:57 INFO  es[][o.e.c.s.ClusterApplierService] new_master {sonarqube}{qT3jesniTNOvQuFTc150Lg}{5wBh0CdCRxqzg0TXK1029Q}{127.0.0.1}{127.0.0.1:9001}{rack_id=sonarqube}, reason: apply cluster state (from master [master {sonarqube}{qT3jesniTNOvQuFTc150Lg}{5wBh0CdCRxqzg0TXK1029Q}{127.0.0.1}{127.0.0.1:9001}{rack_id=sonarqube} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
2019.06.12 16:51:57 INFO  es[][o.e.n.Node] started
2019.06.12 16:51:57 INFO  es[][o.e.g.GatewayService] recovered [0] indices into cluster_state
2019.06.12 16:52:03 INFO  es[][o.e.n.Node] stopping ...
2019.06.12 16:52:03 INFO  es[][o.e.n.Node] stopped
2019.06.12 16:52:03 INFO  es[][o.e.n.Node] closing ...
2019.06.12 16:52:03 INFO  es[][o.e.n.Node] closed

Call me blind, but I cannot see any error why the ES does not keep running. I tried to set ES JVM arguments to bigger memory (1G, 2G) but no success. I checked system logs, nothing in them about ES or any other Java-based process.

Note: I tried to read all documentation about how to install SonarQube, followed different guides with proper restarts, no success. Please do not ask me if I tried to reinstall it or rebuild the whole machine. I’m newbie with SonarQube but not with Linux administration. If I need to do something, please explain it why you think I need to do it because in the end I have to write a system documentation about this and I have to fully understand what’s going on.

Looks like the web process stops before the elasticsearch process

jvm 1    | 2019.06.12 16:51:58 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
jvm 1    | 2019.06.12 16:51:58 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=2, logFilenamePrefix=web]] from [/opt/sonarqube/sonarqube-7.7]: /usr/java/jdk1.8.0_212-amd64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/sonarqube-7.7/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttp.proxyHost=aproxy.some.where -Dhttps.proxyHost=aproxy.some.where -cp ./lib/common/*:/opt/sonarqube/sonarqube-7.7/lib/jdbc/postgresql/postgresql-42.2.5.jar org.sonar.server.app.WebServer /opt/sonarqube/sonarqube-7.7/temp/sq-process5107517876487134023properties
jvm 1    | 2019.06.12 16:52:03 INFO  app[][o.s.a.SchedulerImpl] Process [web] is stopped
jvm 1    | 2019.06.12 16:52:03 INFO  app[][o.s.a.SchedulerImpl] Process [es] is stopped

There should be a web.log file in the logs folder, anything interesting in there?

To try and narrow things down, you might try temporarily removing any proxy configuration, since that is being passed to the web server JVM.

Colin

@Colin This solved my problem. The confusion is made because the console reports [es] exits with code 143 that is clearly not OK, but not reports a failing [web]. This should be improved for better ops experience and quicker troubleshoot. I thought [web] is stopped preventively since it hardly depends on [es] and never guessed [web] is failed.

Thanks for your great assistance and have a nice day.