SonarQube scans failing with intermittent 504 Gateway TimeOut and 502 Bad Gateway errors

SonarQube Server / Community Build
1

SonarQube Version - 10.4.1 (Enterprise Edition) - (Self-Hosted)
Jenkins version - 2.426.3
SonarQube scanner plugin version - 2.17.1

We are using Jenkins to run sonar scans for Java and .Net applications.
Often times we are seeing 504 Gateway Timeout and 502 Bad GateWay errors as shown below
we couldn’t find anything relevant in the logs and they are attached to this ticket.

INFO: Analysis report uploaded in 124ms
INFO: ------------- Check Quality Gate status
INFO: Waiting for the analysis report to be processed (max 900s)
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 2:59.154s
INFO: Final Memory: 60M/207M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
ERROR: Failed to get CE Task status - HTTP code 502:

502 Bad Gateway

502 Bad Gateway

Microsoft-Az ERROR: ERROR: Re-run SonarScanner using the -X switch to enable full debug logging. ERROR: SonarQube scanner exited with non-zero code: 2

Error 504 on
key=ia-esot-dev-dotnet :

504 Gateway Time-out

504 Gateway Time-out

Microsoft-Azure-Application-Gateway/v2

sonarqube_ce.log (2.6 MB)
sonarqube_web.log (3.2 MB)
sonarqube_app.log (446.9 KB)
sonarqube-system-info-985C565C-AYXqSfVpHksT8jeGrbC3-2024-3-8-16-11.json (18.3 KB)

2

Hey there.

These errors are more likely being returned by the Microsoft-Azure-Application-Gateway sitting between the scanner and the SonarQube server. I’m not too familiar with Azure Application Gateways, but if there are logs available at that layer, I suggest starting there.

3

Thanks for the response Colin.
We tried looking for logs at Application Gateway and couldn’t find anything there.
Is there any way we can monitor SonarQube application/URL health check to see if its Gateway dropping connections or application itself?
Do you find anything relevant error messages from the logs attached ?

4

You have a striking number of login failures (search web.log for login failure) and overall a lot of login events in just the logs shared, but that’s pretty normal if you’re running analyses which authenticate each API call.

I did notice this in your system info:

   "Web JVM State": {
        "Max Memory (MB)": 1024,
        "Free Memory (MB)": 71,
        "Heap Committed (MB)": 432,
        "Heap Init (MB)": 128,
        "Heap Max (MB)": 1024,
        "Heap Used (MB)": 359,
        "Non Heap Committed (MB)": 223,
        "Non Heap Init (MB)": 7,
        "Non Heap Used (MB)": 218,
        "System Load Average": "103.0% (last minute)",
        "Threads": 44
    },

That’s not much free memory left for the Web Server. You may want to beef up the memory available to sonar.web.javaOpts (try 2GB)