Health Check Failures in ALB with SonarQube Pod

I am facing an issue where the health checks for my SonarQube pod in an AWS Kubernetes setup with ALB (Application Load Balancer) are failing, even though the pod is running fine and accessible via HTTP. The health checks from the ALB to the SonarQube pod are returning as unhealthy, and I’m unable to identify the root cause.
Environment Setup:

  type: ClusterIP
  externalPort: 9000
  internalPort: 9000
  labels:
  annotations: {}`

ingress config

ingress:
  enabled: true
  # Used to create an Ingress record.
  hosts:
    - name: sonarqube-dev.dmain.com
      # Different clouds or configurations might need /* as the default path
      path: /
      # For additional control over serviceName and servicePort
      #serviceName: 
      # servicePort: somePort
      # the pathType can be one of the following values: Exact|Prefix|ImplementationSpecific(default)
      pathType: Prefix
  annotations: 
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:cer
    alb.ingress.kubernetes.io/group.name: alb-group name
    alb.ingress.kubernetes.io/healthcheck-healthy-threshold: "2"
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: "30"
    alb.ingress.kubernetes.io/healthcheck-path: /api/system/status
    alb.ingress.kubernetes.io/healthcheck-port: "9000"
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "20"
    alb.ingress.kubernetes.io/healthcheck-unhealthy-threshold: "6"
    alb.ingress.kubernetes.io/inbound-cidrs: valid ips
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS": 443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/security-groups: sg
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/target-type: ip
    
  # kubernetes.io/tls-acme: "true"

  # Set the ingressClassName on the ingress record
  ingressClassName: alb

Problem Description:

  • The SonarQube pod is running and accessible within the cluster. However, the ALB health checks are failing.
  • Pod Access: When trying to access /api/system/status directly, it returns a 405 Method Not Allowed, but accessing the root / returns a 200 OK status.
  • ALB Health Check: Despite the pod being accessible via curl -I http://pod-ip:9000/, the health check from the ALB fails, marking the target as unhealthy.
  • Health Check Logs: The ALB health check logs show that the target is unhealthy, even though the pod is functional and the curl command from within the pod works fine.

What I’ve Tried So Far:

  1. Increasing Readiness Delay: Increased the readinessProbe initialDelaySeconds to allow SonarQube enough time to initialize.
  2. Health Check Timeout: Set the healthcheck-timeout-seconds for the ALB target to 30 seconds, but the issue persists.
  3. Idle Timeout: Increased the idle_timeout.timeout_seconds in the ALB configuration to 300 seconds, but still no resolution.
  4. Correct Path in ALB: Verified that the health check path is correctly set to /api/system/status and the port is set to 9000.

Symptoms Observed:

  • Pod Access:
    • curl -I http://localhost:9000/api/system/status returns 405 Method Not Allowed.
    • curl -I http://localhost:9000/ returns 200 OK.
  • ALB Health Check Failure: ALB shows the target as unhealthy despite the pod’s readiness and the ability to access it internally.

Questions:

  1. Why does the health check fail even though the pod responds correctly?
  2. What can be configured or adjusted on the ALB side to ensure that health checks pass?
  3. Is there a discrepancy between the expected health check path and what SonarQube exposes for readiness?
  4. Is there any additional configuration required for the SonarQube pod in relation to health checks and ALB compatibility?

Hey there.

A number of AWS ELB issues have been solved in this community by setting the path to /* rather than /.

Can you give it a try?

[quote=“LTS deployed via helm; Stuck on “Loading…”, post:9, topic:103194”]
I managed to fix my problem by changing my ingress host from / to /*. Thanks
[/quote]

Yes i did it before, but the same problem. The alb target fails the health check eventually.

Hello @Rabeb_Hamed thanks a lot for participating in the community.

Is your problem definitely solved or do you need some more help from our side ?

regards.
Jeremy.

1 Like