- SonarQube 10.3
- Sonar Scanner 6.2.1 (general, no jre included)
- M2 MacOS 14.5
- Xcode 15.2
I am running the scanner on MacOS against a code base that consists of swift, objective-C and C++. The code is compiled with Xcode and we generate compilation database using xcpretty. The file is fed into the scanner and the whole thing has worked reliably for quite some time. This is how we run the scanner:
sonar-scanner -X -Dsonar.coverageReportPaths=$COVERAGE_REPORT_FILE -Dsonar.cfamily.compile-commands=$COMPILATION_DATABASE_FILE -Dsonar.pullrequest.base=$CHANGE_TARGET -Dsonar.pullrequest.branch=$CHANGE_BRANCH -Dsonar.pullrequest.key=$CHANGE_ID
Recently I introduced ccache by redirecting compiler calls through a wrapper scripts and setting CC and CXX environment variable to make xcodebuild use them. This works most of the time but I am hitting a hang during scanner execution about 1 out of 5 times where the scanner times out on the compiler probe. It is important to note that the environment is not shared. Only this process is running on the system at that time as we allocated the entire machine for the agent that has a single executor. So, this is not a resource contention issue.
Dump from a Jenkins job where I see this problem:
21:31:20 01:31:19.993 DEBUG: Probing compiler: [/Users/jenkins1/setup-sdk/scripts/ccache-clang, -x, c, -target, arm64-apple-ios16.0-simulator, -isysroot, /Applications/Xcode-15.2.0.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.2.sdk, -v, -dM, -E, -]
21:31:52 01:31:50.028 INFO: ------------------------------------------------------------------------
21:31:52 01:31:50.028 INFO: EXECUTION FAILURE
21:31:52 01:31:50.028 INFO: ------------------------------------------------------------------------
21:31:52 01:31:50.028 INFO: Total time: 43.885s
21:31:52 01:31:50.062 INFO: Final Memory: 56M/228M
21:31:52 01:31:50.062 INFO: ------------------------------------------------------------------------
21:31:52 01:31:50.062 ERROR: Error during SonarScanner execution
21:31:52 java.lang.IllegalStateException:
21:31:52 The compiler execution timed out, make sure that:
21:31:52 * the analysis is running on appropriate hardware
21:31:52 * eventual compiler license check over network is not taking too much time
21:31:52 The executed process:
21:31:52 [/Users/jenkins1/setup-sdk/.scannerwork/.sonartmp/12156166276171818188/subprocess, -exec, /Users/jenkins1/setup-sdk/scripts/ccache-clang, /Users/jenkins1/setup-sdk/scripts/ccache-clang, -x, c, -target, arm64-apple-ios16.0-simulator, -isysroot, /Applications/Xcode-15.2.0.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.2.sdk, -v, -dM, -E, -]
21:31:52
This is what the ccache-clang wrapper script looks like:
#!/bin/sh
if type -p ccache >/dev/null 2>&1 && [ -z "$CCACHE_DISABLE" ]; then
export CCACHE_SLOPPINESS=modules
export CCACHE_DEPEND=true
exec ccache clang "$@"
else
exec clang "$@"
fi
What is very confusing is that this works most of the time, and about 20% of the time the scanner times out. When the scan succeeds, the compiler probe takes about 5 seconds to run.
I tried running the command manually to see if I can spot something interesting but it always hangs for me if I run it by hand which suggests I am missing something in this scenario, like this:
/Users/jenkins1/setup-sdk/scripts/ccache-clang -x c -target arm64-apple-ios16.0-simulator -isysroot /Applications/Xcode-15.2.0.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.2.sdk -v -dM -E -
I also tried explicitly disabling ccache when running the scanner via CCACHE_DISABLE=true env var so when the compiler probe runs, ccache isn’t called and the compiler is called directly through the wrapper (else clause). That also hangs from time to time.
I could use some guidance on where to go from here and how to debug this further. Thanks!