Why was bash added between 8.0.0.41269 and 8.0.0.41390?

turtleyacht · May 24, 2023, 5:03pm

We found the following error that sometimes causes a timeout in the scanner:

[INFO] SonarQube version: 8.0.0.41390
...
> Task :sonarqube
...
Error while executing the command uname -m Cannot run program "/bin/bash": error=2, No such file or directory
...
ERROR: Timed out waiting for Sonar analysis to complete

However, it seems to happen only when a newer version of SonarQube is pulled down (.41390) versus .41269.

Could you point us to release notes where bash was added as a required dependency?

We are using GitHub Actions to run Sonar in a Docker alpine image against a Java repo built using Gradle. Our workaround has been to install bash into the container.

It would otherwise be benign except that it does fail the GitHub Actions run, and not something the dev teams can address, since it’s not their code changes that caused the failure. Since we’re looking to enforce PRs with Quality Gate, we can’t have false errors otherwise.

On our end, installing bash is a fine workaround, but it’s perceived as a random change; we need a little more details since it would impact all repos.

The scanner command used is

apk add git && /app/gradlew --no-daemon -x cacheSecrets -x loadCachedSecrets --init-script /sonar-init.gradle clean sonarqube  -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=*** -Dsonar.login=*** -Dsonar.branch.name=*** -Dsonar.projectVersion=*** -Dsonar.projectKey=*** -Dsonar.projectName=***

Thank-you.

Colin · May 25, 2023, 8:17am

Hey there.

It’s not clear what’s happening in the logs when you get this error. Can you find out by adding at least (--info level logs. If it’s not clear, then --debug).

turtleyacht · May 25, 2023, 3:35pm

Hi Colin,

I didn’t find much difference using --info or --debug; one noticeable thing versus original log is that the error message appears before > Task :sonarqube, if that helps.

Here are the snippets from both flags:

`--info`

CPD Executor CPD calculation finished (done) | time=21ms
Error while executing the command uname -m Cannot run program "/bin/bash": error=2, No such file or directory

> Task :sonarqube
...

`--debug`

2023-05-25T14:41:04.9693460Z 2023-05-25T14:41:04.917+0000 [LIFECYCLE] [org.sonarqube.gradle.SonarQubeTask] 
2023-05-25T14:41:04.9694262Z 2023-05-25T14:41:04.917+0000 [INFO] [org.sonarqube.gradle.SonarQubeTask] CPD Executor CPD calculation finished (done) | time=61ms
2023-05-25T14:41:05.7474554Z 2023-05-25T14:41:05.684+0000 [DEBUG] [jdk.internal.httpclient.debug] [HttpClient-1-SelectorManager] [51s 69ms] HttpClientImpl(1) next timeout: 0
2023-05-25T14:41:05.7480719Z 2023-05-25T14:41:05.684+0000 [DEBUG] [jdk.internal.httpclient.debug] [HttpClient-1-SelectorManager] [51s 69ms] HttpClientImpl(1) next expired: 0
2023-05-25T14:41:05.7485832Z 2023-05-25T14:41:05.684+0000 [DEBUG] [jdk.internal.httpclient.debug] [HttpClient-1-SelectorManager] [51s 69ms] HttpClientImpl(1) Next deadline is 3000
2023-05-25T14:41:05.9480744Z 2023-05-25T14:41:05.878+0000 [DEBUG] [okhttp3.internal.concurrent.TaskRunner] Q10058 starting              : OkHttp ConnectionPool
2023-05-25T14:41:05.9482145Z 2023-05-25T14:41:05.878+0000 [DEBUG] [okhttp3.internal.concurrent.TaskRunner] Q10058 run again after 300 s : OkHttp ConnectionPool
2023-05-25T14:41:05.9482932Z 2023-05-25T14:41:05.878+0000 [DEBUG] [okhttp3.internal.concurrent.TaskRunner] Q10058 finished run in 759 µs: OkHttp ConnectionPool
2023-05-25T14:41:06.8474756Z 2023-05-25T14:41:06.756+0000 [DEBUG] [okhttp3.internal.concurrent.TaskRunner] Q10058 starting              : OkHttp ConnectionPool
2023-05-25T14:41:06.8475917Z 2023-05-25T14:41:06.756+0000 [DEBUG] [okhttp3.internal.concurrent.TaskRunner] Q10058 run again after 300 s : OkHttp ConnectionPool
2023-05-25T14:41:06.8501883Z 2023-05-25T14:41:05.036+0000 [ERROR] [org.sonarqube.gradle.SonarQubeTask] Error while executing the command uname -m Cannot run program "/bin/bash": error=2, No such file or directory
2023-05-25T14:41:06.8502665Z 2023-05-25T14:41:06.756+0000 [DEBUG] [okhttp3.internal.concurrent.TaskRunner] Q10058 finished run in 229 µs: OkHttp ConnectionPool
2023-05-25T14:41:06.8503582Z 2023-05-25T14:41:04.845+0000 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] 
2023-05-25T14:41:06.8504202Z 2023-05-25T14:41:04.845+0000 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] > Task :sonarqube

We are using org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:3.3 in the sonar-init.gradle file.

Do these help; what additional information should I provide?

If available, would I be able to see a diff between release 8.0.0.41269 and 8.0.0.41390?

Colin · May 26, 2023, 8:57am

Thanks.

In fact… there should not have been any change. We have called uname -m for a long time to collect analytics information like the architecture being used.

Is this a self-hosted runner? A specific docker image you’re using as your base (maybe something got updated on that layer?)

turtleyacht · May 31, 2023, 1:15am

Thank-you, @Colin. I’m still looking into this, although I’m running out of ideas.

I saw that gradlew calls uname twice, but not uname -m.

We have called uname -m for a long time to collect analytics

Could you point me to where that happens, if possible?

Does it happen in the org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:3.3 .jar?

Although I understand it’s not the cause of this issue, since it has been around for some time (and our Sonar scripts have worked for over a year otherwise).

One hindrance to just installing bash in the Alpine container is that we use docker run --rm ..., which deletes itself between invocations. So we can’t cache it, and–for our purposes–precious seconds count on the self-hosted runners, to avoid blocking multiple teams’ builds.

For the same reason, we may not be able to switch to a non-Alpine image, since installing bash there takes, for example, increases the GitHub Actions workflow by a minute.

Still looking…

Similar post, but resolution was just to install bash

Colin · May 31, 2023, 9:54am

This happens in closed source code, so I can’t share it.

Here in your logs I see clearly the uname -m. What else are you seeing?

Can you tell us what image you are using? Maybe we can reproduce it on our side, or track down a change that got made causing this.

turtleyacht · May 31, 2023, 8:15pm

I set up a (somewhat minimal) repro. May I send it to an email address you have for that?

Once the .zip is unzipped, cd in and execute

docker run -it --rm -v $(PWD):/app -v $(PWD)/sonar-init.gradle:/sonar-init.gradle -w /app amazoncorretto:11-alpine sh

Once inside the container, then run

SONAR_TOKEN=*** ./gradlew clean sonarqube --no-daemon --init-script /sonar-init.gradle -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=test-org

(Where SONAR_TOKEN and sonar.organization are supplied on your end.)

The output should be similar to

turtleyacht · June 2, 2023, 7:16pm

Hi @Colin,

Was my last request okay? If email is not preferred, should I make it available from GitHub (for example)?

I otherwise do not see an option to attach files in the thread.

Just to follow up on your replies,

This happens in closed source code, so I can’t share it.

That’s okay. We have a repro .zip that we can send over, or I can post it to GitHub if you prefer.

Here in your logs I see clearly the uname -m . What else are you seeing?

That’s pretty much all we have to go on. The repro .zip I want to share should hopefully help the team investigate further on their end.

Can you tell us what image you are using? Maybe we can reproduce it on our side, or track down a change that got made causing this.

It is reproducible using the code in the repro, mounted as a volume into the amazoncorretto:11-alpine Docker image.

Colin · June 5, 2023, 7:36am

I’ve pinged some experts who can take a closer look.

turtleyacht · June 14, 2023, 3:04pm

Hi @Colin, any update? Just wanted to check in.

Thank-you,
Chris

turtleyacht · June 21, 2023, 3:52pm

Hi @Colin, did you want us to go ahead and put up a repo with the reproducible issue?

Anything we can do to help on our end?

Thank-you,
Chris

Csaba_Feher · June 29, 2023, 11:17am

Hi @turtleyacht ,

Sorry for the long wait. Could you please send the reproducer zip to csaba.feher@sonarsource.com?
I will have a look and get back to you.

Thanks
Csaba

turtleyacht · July 6, 2023, 9:37pm

Hi @Csaba_Feher,

No worries! Thank-you for the email. Sent along the zip.

Thank-you,
Chris

turtleyacht · July 6, 2023, 10:05pm

@Csaba_Feher, I sent the .zip but received an error email with this message:

550 5.0.350 One or more of the attachments in your email is of a file type that is NOT allowed by the recipient’s organization.

Could this be an issue on your end? Should we try a private GitHub repo or something else?

Thank-you,
Chris

turtleyacht · July 7, 2023, 4:01pm

Thank-you @Csaba_Feher, sent it along via the method suggested.

Chris

Csaba_Feher · July 12, 2023, 2:34pm

Hi @turtleyacht

I have checked the reproducer, and it is really helpful, we are investigating the possible options will get back to you in the upcoming days.

Thanks
Csaba

Csaba_Feher · July 19, 2023, 1:40pm

Hi @turtleyacht,

We are working on a fix which should solve the problem for you. It solved the issue in our tests and using the reproducer you provided.
I will let you know as soon as it is released, and after please have a try.

Thanks
Csaba

Csaba_Feher · August 2, 2023, 2:17pm

@turtleyacht

The fix should have been merged; I was wondering if you could retry and if it is working for you.

Thanks
Csaba

turtleyacht · August 3, 2023, 6:25pm

Thank-you, @Csaba_Feher! I’m not seeing the error in recent logs now, although I am verifying with a manual run.

What ended up being the fix?

Csaba_Feher · August 11, 2023, 1:52pm

Hi Christopher,

We applied a quick fix and we might make a hardening on the space that caused the issue. Unfortunately, I cannot share implementation details on our private source code.
Please let me know if any problem arises with the feature.

Thanks
Csaba