Fail to get bootstrap index from server: timeout

I just updated our server from 6.7.6 to 7.9.1 and am now running into this error.
Only difference is I am running Gradle (Sonar plugin v2.7.1) instead of Maven and GitLab CI instead of Jenkins.
Before the update, everything worked fine and there was no change in network configuration.
Just like Fernandes, I can successfully wget from the docker container.
When running the same build from my (Windows) workstation, it works fine.
I have attached the error including stack trace.

sonar_stacktrace.txt (10.6 KB)

Hi,
Could it be that the connection from GitLab CI is slower and timing out?
Did you try doing the wget from GitLab as well? If so, did it take a long time even if it was successful?
There isn’t much we can do to troubleshoot or fix network timeout issues.

What I can tell from the logs is that it’s failing, with a timeout, the very first connection that the scanner tries to establish with the server. The information read with this connection is only a few bytes long, so it’s probably a connect timeout rather than a read timeout.

Hi Duarte
Yes, I tried wget as well. It was successfull and near instant, as it should be. I very much doubt the issue is caued by the network, because as I said, the exact same setup worked fine with SQ 6.7.6. Between the versions, was there a change in network code like forcing https, support for IPv6, rejecting older protocols, etc.?

So to be clear. I tried to wget the webroot, i.e. myserver.com:9000. What is the exact URL that is queried in that first request? I want to wget that one.

I investigated some more. So, looking at the logs when running the (working) build from my machine, it seems the first request goes to /batch/index. When I try to wget that URL, the connection is established, the request arrives, the server responds with a 200 (same as in build) but wget doesn’t terminate within reasonable time. Here’s the wget output (hostname and IP changed):

--2019-10-09 07:36:33--  http://myserver.com:9000/batch/index
Resolving myserver.com (myserver.com)... 1.2.3.4
Connecting to myserver.com (myserver.com)|1.2.3.4|:9000... connected.
HTTP request sent, awaiting response...

OK, weird, at least the server gets and logs the request. I also tried to wget /batch_bootstrap/index which again worked fine.
When browsing to /batch/index with Firefox from my workstation, I get the (seemingly) normal response sonar-scanner-engine-shaded-7.9.1-all.jar|a1c05423113146c25aa5a0819888ceec

Could it be that the response is missing some terminator?

BTW, is it normal that the user string of the request is “ScannerGradle/2.8-SNAPSHOT/4.9”, even if I am running plugin v2.7.1?

I also tried with Gradle 5.6.2 and got the same results.

Hi,

You’re right, the first request is batch/index.
The string of the request doesn’t look right - I will look into it. In any case it shouldn’t affect the response of that WS, which is confirmed with wget.

Are you using a public docker image with GitLab, so that I could try to reproduce it?

Yes, I am using the gradle:jdk11 image (see here). However, I am not actually using the Gradle install of the image but instead using Gradle Wrapper to download the distro (gradle-4.9-all but also reproduced with gradle-5.6.2-all).
One thing that I have yet to try is using an image with a different JDK.

I’ve just done a test with GitLab CI, using gradle:jdk11, sonarqube plugin 2.7.1 and SonarQube 7.9.1 CE. Everything worked as expected.
Is there anything particular about your setup of the server or gitlab yml that could make a difference?

Well yes, absolutely: Company-hosted GitLab, company proxy, and company-shared runners to name a few.
The reason I don’t suspect any of those to be causing the problem is that nothing about them changed. The only change that could have caused the problem was the SQ Server update and its new JDK.

Since my last comment, I have also tried this with a different docker image that has a different JDK (maven:3-jdk-11). The results were the same.

What I may do is run Wireshark on the SQ server to check the outgoing packages. Unfortunately, I can’t do the same on the client-machine, as it’s not under my control (company shared runner).

What did you mean when you said the string of the request didn’t look right?

After some hours of debugging, I finally found out that somewhere on our network, HTTP responses with the header "Sonar-Version" are blocked. This is very specific, as different capitalizations of the header are not blocked. I have no idea why this filter exists. Did this header exist before? Maybe with a different capitalization?

My workaround was to put SonarQube behind a HTTPS reverse proxy.

1 Like

Thanks for letting us know.
I believe that header was introduced with this ticket in v7.2.