SonarQube service is stopped: connection refused [es] error 143

dfavato · August 11, 2019, 11:21pm

SonarQube version: 7.9.1

I’m able to start the SonarQube without issues running the following command:

sudo /usr/share/webapps/sonarqube/bin/linux-x86-64/sonar.sh start

Howerver, when I try to start the sonarqube.service which the unit file is:

[Unit]
Description=An open source platform for continuous inspection of code quality
Documentation=http://www.sonarqube.org
Wants=network.target
After=network.target postgresql.service
Requires=postgresql.service

[Service]
Type=simple
User=sonarqube
Group=sonarqube
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=sonarqube
RuntimeDirectory=sonarqube
LimitNOFILE=65536
LimitNPROC=4096
ExecStart=/usr/share/webapps/sonarqube/bin/linux-x86-64/sonar.sh start
ExecStop=/usr/share/webapps/sonarqube/bin/linux-x86-64/sonar.sh stop

[Install]
WantedBy=multi-user.target

The service starts but a few seconds later stops

ago 11 20:11:12 dell5570 systemd[1]: Started An open source platform for continuous inspection of code quality.
ago 11 20:11:12 dell5570 sonarqube[11861]: Starting SonarQube...
ago 11 20:11:12 dell5570 sonarqube[11861]: Started SonarQube.
ago 11 20:11:12 dell5570 sonarqube[11954]: Gracefully stopping SonarQube...
ago 11 20:11:17 dell5570 sonarqube[11954]: Stopped SonarQube.

The log files show a connection refused type of error:

org.elasticsearch.transport.ConnectTransportException: [][127.0.0.1:9001] connect_exception
        at org.elasticsearch.transport.TcpTransport$ChannelsConnectedListener.onFailure(TcpTransport.java:1309)
        at org.elasticsearch.action.ActionListener.lambda$toBiConsumer$2(ActionListener.java:100)
        at org.elasticsearch.common.concurrent.CompletableContext.lambda$addListener$0(CompletableContext.java:42)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
        at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
        at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
        at org.elasticsearch.common.concurrent.CompletableContext.completeExceptionally(CompletableContext.java:57)
        at org.elasticsearch.transport.netty4.Netty4TcpChannel.lambda$new$1(Netty4TcpChannel.java:72)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:327)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:343)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:591)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:508)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Conexão recusada: /127.0.0.1:9001
        at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
        at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:327)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
        ... 6 common frames omitted
Caused by: java.net.ConnectException: Conexão recusada
        ... 10 common frames omitted
2019.08.11 20:11:14 DEBUG app[][o.s.a.e.EsConnectorImpl] Connected to Elasticsearch node: [127.0.0.1:9001]
2019.08.11 20:11:15 DEBUG app[][o.s.a.SchedulerImpl] Stopping [ce]...
2019.08.11 20:11:15 DEBUG app[][o.s.a.SchedulerImpl] Stopping [web]...
2019.08.11 20:11:15 DEBUG app[][o.s.a.SchedulerImpl] Stopping [es]...
2019.08.11 20:11:15 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
2019.08.11 20:11:15 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 143
2019.08.11 20:11:15 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

What is wrong with my unit file? Why does I can run it without issues when starting it directly from the command line?

This is the output of cat /usr/share/webapps/sonarqube/bin/linux-x86-64/sonar.sh | grep "RUN_AS_USER"

RUN_AS_USER=sonarqube
    if [ "X$RUN_AS_USER" != "X" ]
        if [ "`$IDEXE -u -n`" = "$RUN_AS_USER" ]
            RUN_AS_USER=""
    if [ "X$RUN_AS_USER" != "X" ]
        # If LOCKPROP and $RUN_AS_USER are defined then the new user will most likely not be
                RUN_AS_GROUP=`groups $RUN_AS_USER | awk '{print $3}' | tail -1`
                    RUN_AS_GROUP=$RUN_AS_USER
                chown $RUN_AS_USER:$RUN_AS_GROUP $LOCKFILE
        su -m $RUN_AS_USER -c "\"$REALPATH\" $2"

ganncamp · August 13, 2019, 1:26pm

Hi,

Please check your server logs for errors.

Ann

dfavato · August 15, 2019, 10:30pm

G Ann,
Thank you for your support.

After trying a lot of things I found the solution.

The problem was the Type=simple in my sonarqube.service unit file. I had to change it to forking

Type=forking

It seems like the sonar.sh command forks the current process and exits. As systemd thought that it was a simple service, after returning it executes ExecStop. So setting the service as forking would prevent systemd to issue ExecStop after sonar.sh has successfully started.

In the end the error 143 connection refused had nothing to do with the service stopping. As I should have guessed from the WARN log entry.

ganncamp · August 16, 2019, 11:41am

Hi,

Thanks for checking back in and reporting your solution. I’m glad you got this worked out!

Ann

Eric_Hartmann · August 19, 2019, 9:43am

Hi @dfavato,

If you use the sonar.sh script you have to use Type=forking.
However, I would suggest to use directly the jar as described here https://docs.sonarqube.org/latest/setup/operate-server/#header-3 and keep the type to simple.

Cheers,

dfavato · August 20, 2019, 4:21pm

Eric,

I thought about that. For me the downside of calling the jar directly is that I would have to update de unit file on every update as the jar file has the version in its name.

However, I’m new into making unit files so I don’t know what are the consequences of setting the type as forking and using sonar.sh.