I’m trying to set a new project under SonarQube Server (Enterprise Edition) 10.7 hosted on Gitlab SAAS, however every time there’s a Project Analysis background job launched, it takes hours to complete.
I turned on the DEBUG logs, and have been able to see the following:
2024.12.04 15:16:16 DEBUG ce[2e265af7-96db-4cec-9bea-acbbf73a5a4b][o.s.w.c.OkHttpClientBuilder] <-- HTTP FAILED: java.net.SocketTimeoutException: timeout
2024.12.04 15:16:16 WARN ce[2e265af7-96db-4cec-9bea-acbbf73a5a4b][c.sonar.gitlab.G.I] Cannot access GitLab to get vulnerabilities for project <REDACTED>. Error: java.net.SocketTimeoutException: timeout
2024.12.04 15:16:16 DEBUG ce[2e265af7-96db-4cec-9bea-acbbf73a5a4b][c.sonar.gitlab.G.I] Cannot access GitLab to get vulnerabilities for project <REDACTED>. Error: java.net.SocketTimeoutException: timeout
2024.12.04 15:16:16 WARN ce[2e265af7-96db-4cec-9bea-acbbf73a5a4b][c.sonar.gitlab.x.I] Impossible to perform synchronization of changed Gitlab vulnerabilities with SonarQube issues
This happens every time, and only differs on how much time it takes to fail (on this specific execution it failed after around 40k calls to Gitlab.com GraphQL similar to the following:
This seems to be preventing the analysis from completing with success given I still get the information that the project still indicated The main branch of this project is empty., as per the following message in the logs as well:
2024.12.04 15:16:45 DEBUG ce[2e265af7-96db-4cec-9bea-acbbf73a5a4b][c.z.h.p.ProxyConnection] HikariPool-1 - Executed rollback on connection org.postgresql.jdbc.PgConnection@647dac1e due to dirty commit state on close().
Questions:
What are those calls to Gitlab.com GraphQL doing? (this slows the background analysis in hours… a lot of hours)
Can the SocketTimeoutException being raised due to Gitlab “Authenticated API traffic for a user” (a support case has also been raised on Gitlab due to the amount of requests being executed given being partners with Sonar and the error not being clear on any side)
Obviously, the timeout is where we need to start. I’m glad to see that you’ve raised a support ticket on the GitLab side. While we wait for them to get back to you, I think it might be helpful to talk to your network folks to ask them to trace the traffic: can we make sure the calls are actually getting out to GitLab?
I have also confirmed the requests are reaching Gitlab.com (we get a 200 response) and checked on our load balancer that we are getting traffic (also, the Merge Request decoration is working meaning the requests can access Gitlab.com)
All the other projects already configured do not face this issue, only this one.
GitLab has confirmed that the calls for this project are received and successful 200? Then… the problem has to be on the way back from GitLab. I think it’s time to talk to your network folks.
I was now able to validate the reason for the issue (and it’s a complex one, so bare with me) and I would consider this a bug (although it is a corner case).
For this to be replicated (which I was able to do it in another project) the following conditions must be in place:
Have a project that have previously been able to do a Project Analysis scan
Have the binding setup so it is able to decorate the merge requests
Have the SAST report downloaded at least once to enable the feature to sync the Vulnerabilities between (in this case) Gitlab and SonarQube server
After this has been setup, we noticed that the SonarScanner was taking a long time to run during the CI/CD pipelines and attempted to improve it’s performance as well as the generated data by adding the sonar.tests configuration to the sonar-project.properties file.
This change had an unexpected impact (and not well documented at Analysis parameters where it does not say what value sonar.sources it gets when the sonar.tests property is set) where we noticed that the only code being analysed was the tests.
After setting the sonar.sources properties, we were back to get all the code in SonarQube Server as well as the sync of the Vulnerabilities with Gitlab working fine, in a couple of minutes.
This observations seem to point that SonarQube Server was in a loop attempting to sync the Vulnerabilities but failing to do so because there was no source code to attach it to, although Gitlab had reports associated with it, and would only stop when (for some reason and here I guess randomness was the key given it would provide the exception at different times) the SocketTimeoutException was raised.
I hope this makes sense, and also helps others to identity this very hard to detect issue.
As far as I can see in the code, there is only one call made to GitLab GraphQL API to retrieve vulnerabilities so we can match their statuses in SQS (SonarQube Server).
Technically, there is pagination involved, so there might be a few calls being made, but I’m very surprised by these 40k calls in your report. I would mean that you have several hundreds of thousands of vulnerabilities recorded in GitLab. Is that the case?
This change had an unexpected impact (and not well documented at Analysis parameters where it does not say what value sonar.sources it gets when the sonar.tests property is set) where we noticed that the only code being analyzed was the tests.
The documentation page for the analysis scope is Analysis scope, where it says:
If you are not using Maven, Gradle or .NET then
By default, sonar.sources is set to the current working directory (the path .).
By default, sonar.tests is not set.
My assumption is that by not defining sonar.sources initially, you analyzed way too much code (source, tests, libraries, generated code, etc.), which likely resulted in many vulnerabilities, and issues in general. Setting it up to the correct values solves the problem by simply lowering the volume.
You can validate this theory by also checking at the number of issues and LOC analyzed without proper sonar.sources and sonar.tests, then with proper configuration.