DCE app node not starting up in k8s

SonarQube Server / Community Build
1

Hi, we are integrating SQ 9.0 DCE and while waiting for full license to materialize, so posting questions here before we get the full license.

When I start app nodes in Kubernetes, I created a service for the app nodes, with only one app node as suggested earlier. I see that the app node is failing to start up, logs is

2021.07.27 03:26:19 ERROR app[][c.h.i.c.i.o.JoinMastershipClaimOp] [xx.xx.xx.xx]:9003 [SonarQube] [4.2] Target is this node! -> [xx.xx.xx.xx]:9003

java.lang.IllegalArgumentException: Target is this node! -> [xx.xx.xx.xx]:9003

at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.checkTarget(OutboundResponseHandler.java:226)

at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendNormalResponse(OutboundResponseHandler.java:125)

at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendResponse(OutboundResponseHandler.java:101)

at com.hazelcast.spi.impl.operationservice.Operation.sendResponse(Operation.java:475)

at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:282)

at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)

at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:469)

at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)

at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)

at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)

at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
2

Hi @tlong ,

i can only repeat the answer that i already gave you in other threads you opened.
Hazelcast is the main reason why we do not officially support running the DCE on k8s.

We are actively working on fixing this and hope that we can improve the situation by the next release and go into beta with the DCE on k8s support with the delivery of MMF-2412. you are more than welcome to have a look at our work then and give feedback what we could improve there.

as for your current setup and the issue that you are facing; can you elaborate your complete setup a little more detailed? specially from a configuration and network stand it would be great to understand your layout

3

Thanks Tobias, Hazelcast not very stable in k8s, I looked up online, looks like it is called “split-brain”. I think if there are exposed ways to declare an app node as web leader and avoid the confusion in hazelcast, that might work nicely.

4

There are other mechanism as well to prevent a split brain scenario. again we are working on this, so stay tuned :slight_smile: