On large (1M) project: java.lang.IllegalStateException: Fail to request */api/ce/submit

Since about a week we get the error message below. Other analysis on SonarCloud connecting from the same machine work as expected. Only for this single project, which is roughly 1M loc, we continuously see the error below, always at the same point, always the same error. Any suggestions on where to look for answers?
We use Azure DevOps, self-hosted agents.

2022-08-31T13:35:46.3589759Z INFO: SCM Publisher 528/528 source files have been analyzed (done) | time=24274ms
2022-08-31T13:35:46.9168441Z INFO: CPD Executor 215 files had no CPD blocks
2022-08-31T13:35:46.9168912Z INFO: CPD Executor Calculating CPD for 2376 files
2022-08-31T13:35:48.4098106Z INFO: CPD Executor CPD calculation finished (done) | time=1501ms
2022-08-31T13:35:51.6879542Z INFO: Analysis report generated in 2627ms, dir size=91 MB
2022-08-31T13:38:11.2208787Z INFO: Analysis report compressed in 139518ms, zip size=26 MB
2022-08-31T13:38:24.5692259Z INFO: Closing heartbeat service
2022-08-31T13:38:31.0988880Z INFO: ------------------------------------------------------------------------
2022-08-31T13:38:31.0989667Z INFO: EXECUTION FAILURE
2022-08-31T13:38:31.0990096Z INFO: ------------------------------------------------------------------------
2022-08-31T13:38:31.0990453Z INFO: Total time: 9:23.767s
2022-08-31T13:38:31.4579983Z INFO: Final Memory: 305M/1030M
2022-08-31T13:38:31.4580481Z INFO: ------------------------------------------------------------------------
2022-08-31T13:38:31.4586951Z ##[error]ERROR: Error during SonarScanner execution
2022-08-31T13:38:31.4588407Z ERROR: Error during SonarScanner execution
2022-08-31T13:38:31.4590429Z ##[error]java.lang.IllegalStateException: Fail to request https://sonarcloud.io/api/ce/submit?organization=REPLACED&projectKey=ABC&projectName=ABC&characteristic=branch%3DSonarQubeAnalysis&characteristic=branchType%3DLONG
2022-08-31T13:38:31.4592683Z java.lang.IllegalStateException: Fail to request https://sonarcloud.io/api/ce/submit?organization=REPLACED&projectKey=ABC&projectName=ABC&characteristic=branch%3DSonarQubeAnalysis&characteristic=branchType%3DLONG
2022-08-31T13:38:31.4594439Z ##[error]at org.sonarqube.ws.client.HttpConnector.doCall(HttpConnector.java:186)
	at org.sonarqube.ws.client.HttpConnector.post(HttpConnector.java:141)
	at org.sonarqube.ws.client.HttpConnector.call(HttpConnector.java:100)
2022-08-31T13:38:31.4595747Z 	at org.sonarqube.ws.client.HttpConnector.doCall(HttpConnector.java:186)
2022-08-31T13:38:31.4596492Z 	at org.sonarqube.ws.client.HttpConnector.post(HttpConnector.java:141)
2022-08-31T13:38:31.4596937Z 	at org.sonarqube.ws.client.HttpConnector.call(HttpConnector.java:100)
2022-08-31T13:38:31.4597846Z ##[error]at org.sonar.scanner.bootstrap.DefaultScannerWsClient.call(DefaultScannerWsClient.java:59)
2022-08-31T13:38:31.4598811Z 	at org.sonar.scanner.bootstrap.DefaultScannerWsClient.call(DefaultScannerWsClient.java:59)
2022-08-31T13:38:31.4599716Z ##[error]at org.sonar.scanner.report.ReportPublisher.upload(ReportPublisher.java:222)
2022-08-31T13:38:31.4602024Z 	at org.sonar.scanner.report.ReportPublisher.upload(ReportPublisher.java:222)
2022-08-31T13:38:31.4604038Z ##[error]at org.sonar.scanner.report.ReportPublisher.execute(ReportPublisher.java:159)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:414)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2022-08-31T13:38:31.4605616Z 	at org.sonar.scanner.report.ReportPublisher.execute(ReportPublisher.java:159)
2022-08-31T13:38:31.4606141Z 	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:414)
2022-08-31T13:38:31.4606669Z 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2022-08-31T13:38:31.4607308Z 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2022-08-31T13:38:31.4608623Z ##[error]at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:130)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2022-08-31T13:38:31.4609653Z 	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:130)
2022-08-31T13:38:31.4610235Z 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2022-08-31T13:38:31.4611155Z ##[error]at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2022-08-31T13:38:31.4612007Z 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2022-08-31T13:38:31.4612835Z ##[error]at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:58)
2022-08-31T13:38:31.4613615Z 	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:58)
2022-08-31T13:38:31.4614356Z ##[error]at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:52)
2022-08-31T13:38:31.4615158Z 	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:52)
2022-08-31T13:38:31.4616082Z ##[error]at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
2022-08-31T13:38:31.4617025Z 	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
2022-08-31T13:38:31.4617929Z ##[error]at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-08-31T13:38:31.4618697Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-08-31T13:38:31.4619629Z ##[error]at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-08-31T13:38:31.4620537Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-08-31T13:38:31.4621526Z ##[error]at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-08-31T13:38:31.4622434Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-08-31T13:38:31.4623255Z ##[error]at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2022-08-31T13:38:31.4623985Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2022-08-31T13:38:31.4624928Z ##[error]at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
2022-08-31T13:38:31.4625848Z 	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
2022-08-31T13:38:31.4626684Z ##[error]at com.sun.proxy.$Proxy0.execute(Unknown Source)
2022-08-31T13:38:31.4627353Z 	at com.sun.proxy.$Proxy0.execute(Unknown Source)
2022-08-31T13:38:31.4628714Z ##[error]at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
2022-08-31T13:38:31.4629870Z 	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
2022-08-31T13:38:31.4630380Z 	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
2022-08-31T13:38:31.4630840Z 	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
2022-08-31T13:38:31.4631607Z ##[error]at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
2022-08-31T13:38:31.4632455Z 	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
2022-08-31T13:38:31.4633162Z ##[error]at org.sonarsource.scanner.cli.Main.main(Main.java:61)
2022-08-31T13:38:31.4633945Z 	at org.sonarsource.scanner.cli.Main.main(Main.java:61)
2022-08-31T13:38:31.4634592Z ##[error]Caused by: java.net.SocketTimeoutException: timeout
2022-08-31T13:38:31.4635261Z Caused by: java.net.SocketTimeoutException: timeout
2022-08-31T13:38:31.4636006Z ##[error]at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143)
2022-08-31T13:38:31.4636781Z 	at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143)
2022-08-31T13:38:31.4637549Z ##[error]at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:162)
2022-08-31T13:38:31.4638367Z 	at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:162)
2022-08-31T13:38:31.4639100Z ##[error]at okio.AsyncTimeout$sink$1.write(AsyncTimeout.kt:338)
2022-08-31T13:38:31.4639899Z 	at okio.AsyncTimeout$sink$1.write(AsyncTimeout.kt:338)
2022-08-31T13:38:31.4640693Z ##[error]at okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.kt:235)
2022-08-31T13:38:31.4641514Z 	at okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.kt:235)
2022-08-31T13:38:31.4642248Z ##[error]at okio.RealBufferedSink.write(RealBufferedSink.kt:145)
2022-08-31T13:38:31.4643077Z 	at okio.RealBufferedSink.write(RealBufferedSink.kt:145)
2022-08-31T13:38:31.4643951Z ##[error]at okhttp3.internal.http1.Http1ExchangeCodec$KnownLengthSink.write(Http1ExchangeCodec.kt:271)
2022-08-31T13:38:31.4644845Z 	at okhttp3.internal.http1.Http1ExchangeCodec$KnownLengthSink.write(Http1ExchangeCodec.kt:271)
2022-08-31T13:38:31.4645955Z ##[error]at okio.ForwardingSink.write(ForwardingSink.kt:29)
	at okhttp3.internal.connection.Exchange$RequestBodySink.write(Exchange.kt:218)
2022-08-31T13:38:31.4646848Z 	at okio.ForwardingSink.write(ForwardingSink.kt:29)
2022-08-31T13:38:31.4647289Z 	at okhttp3.internal.connection.Exchange$RequestBodySink.write(Exchange.kt:218)
2022-08-31T13:38:31.4648140Z ##[error]at okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.kt:235)
2022-08-31T13:38:31.4648950Z 	at okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.kt:235)
2022-08-31T13:38:31.4649692Z ##[error]at okio.RealBufferedSink.writeAll(RealBufferedSink.kt:184)
2022-08-31T13:38:31.4650532Z 	at okio.RealBufferedSink.writeAll(RealBufferedSink.kt:184)
2022-08-31T13:38:31.4651334Z ##[error]at okhttp3.RequestBody$Companion$asRequestBody$1.writeTo(RequestBody.kt:167)
2022-08-31T13:38:31.4652155Z 	at okhttp3.RequestBody$Companion$asRequestBody$1.writeTo(RequestBody.kt:167)
2022-08-31T13:38:31.4652980Z ##[error]at okhttp3.MultipartBody.writeOrCountBytes(MultipartBody.kt:157)
2022-08-31T13:38:31.4653767Z 	at okhttp3.MultipartBody.writeOrCountBytes(MultipartBody.kt:157)
2022-08-31T13:38:31.4654535Z ##[error]at okhttp3.MultipartBody.writeTo(MultipartBody.kt:93)
2022-08-31T13:38:31.4655291Z 	at okhttp3.MultipartBody.writeTo(MultipartBody.kt:93)
2022-08-31T13:38:31.4656128Z ##[error]at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:59)
2022-08-31T13:38:31.4657097Z 	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:59)
2022-08-31T13:38:31.4657994Z ##[error]at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4658827Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4659766Z ##[error]at org.sonarqube.ws.client.OkHttpClientBuilder.addHeaders(OkHttpClientBuilder.java:201)
2022-08-31T13:38:31.4660621Z 	at org.sonarqube.ws.client.OkHttpClientBuilder.addHeaders(OkHttpClientBuilder.java:201)
2022-08-31T13:38:31.4661511Z ##[error]at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4662389Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4663640Z ##[error]at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4664799Z 	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
2022-08-31T13:38:31.4665316Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4666156Z ##[error]at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
2022-08-31T13:38:31.4667023Z 	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
2022-08-31T13:38:31.4667878Z ##[error]at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4668718Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4669604Z ##[error]at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
2022-08-31T13:38:31.4670429Z 	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
2022-08-31T13:38:31.4671334Z ##[error]at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4672231Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4673155Z ##[error]at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
2022-08-31T13:38:31.4674082Z 	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
2022-08-31T13:38:31.4675645Z ##[error]at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
2022-08-31T13:38:31.4676945Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
2022-08-31T13:38:31.4677473Z 	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
2022-08-31T13:38:31.4677983Z 	at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
2022-08-31T13:38:31.4678891Z ##[error]at org.sonarqube.ws.client.HttpConnector.doCall(HttpConnector.java:184)
	... 25 more
ERROR:
2022-08-31T13:38:31.4679735Z 	at org.sonarqube.ws.client.HttpConnector.doCall(HttpConnector.java:184)
2022-08-31T13:38:31.4680078Z 	... 25 more
2022-08-31T13:38:31.4680275Z ERROR: 
2022-08-31T13:38:32.2070379Z ##[error]The SonarScanner did not complete successfully
2022-08-31T13:38:32.2071348Z The SonarScanner did not complete successfully
2022-08-31T13:38:32.2072812Z ##[error]13:38:32.2  Post-processing failed. Exit code: 1

Hi,

Typically when we see this, it’s something on your network, e.g. a proxy, that’s blocking the analysis report submission based on file size. You should check with your network folks.

 
HTH,
Ann

Hi Ann,

I’ve been doing quite a number of additional runs, to find out why it sometimes works, and sometimes it does not work. We have one build server machine, on which we run 3 Azure DevOps build agents. On the machine, we set the sonarcloud cache to s:_w\sonar
What I noticed is that the build (with sonarcloud) runs good if I clean the SonarCloud cache folder. So with a cleaned cache it works. If I then run it again, on the same agent, I get the error from my previous message. If I clean the cache, it works again. Successful run, with all the updates in SonarCloud.

Is there a way to either clean the cache in between, or figure out why it works with an empty cache, but not with a filled cache?

Hi,

This is very interesting. Thanks for the investigations and the report! I’m going to flag this for more expert attention.

 
Ann

1 Like

Hi @zaat

Could you please enable the Debug logs on the SonarScanner (using -Dsonar.verbose=true or /d:sonar.verbose=true in the SonarScanner command line) and send me the full logs of a failed build?

The result will be huge, any file-sharing tool suitable for you will be fine. If you want to save the burden of removing private data from the logs, I can start a private thread with you.

Hi @Claire_Villard ,

Apologies for the late reply. We have worked through our companies network issues, as after a few days more teams reported similar issues (not only the large project) using the same build servers. Currently it looks like our company’s anti-virus proxy software is causing the irregularities that we see (the error mentioned earlier). If the rules on the proxy are switched off, it works. Will keep you posted, as this could be interesting information others might be facing as well.

1 Like

For sure, any information about the issue and how to fix will be super valuable! Thanks for contributing to the community!