Cannot run program clang in directory

ivarorn · October 30, 2019, 11:42am

Hi…

I am trying to run a pretty basic Objective-C demo project analysis in Azure DevOps.
My project builds but when I try to run the scanner I get this error.
The scan seems to go pretty far but errors out when probing the compiler.

11:31:59.512 DEBUG: Probing compiler: [/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang, -x, objective-c, --std, gnu11, -target, arm64-apple-ios12.4, -isysroot, /Applications/Xcode_11.1.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.1.sdk, -v, -dM, -E, -]

I execute the scanner using this command: /Users/runner/runners/2.159.2/work/1/s/SonarScanner/bin/sonar-scanner -X -Dproject.settings=/Users/runner/runners/2.159.2/work/1/s/sonar-project.properties

My config file is set up like this:

sonar.projectKey=<project key>

sonar.projectName=<project name>

sonar.projectVersion=1.0

sonar.sources=.

sonar.cfamily.build-wrapper-output=build_wrapper_output_directory

sonar.sourceEncoding=UTF-8

sonar.host.url=https://sonarcloud.io

sonar.login=<login token>

Any ideas what I might be configuring the wrong way ?

11:31:59.512 DEBUG: Probing compiler: [/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang, -x, objective-c, --std, gnu11, -target, arm64-apple-ios12.4, -isysroot, /Applications/Xcode_11.1.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.1.sdk, -v, -dM, -E, -]

11:31:59.539 INFO: ------------------------------------------------------------------------

11:31:59.539 INFO: EXECUTION FAILURE

11:31:59.539 INFO: ------------------------------------------------------------------------

11:31:59.539 INFO: Total time: 28.786s

11:31:59.586 INFO: Final Memory: 26M/94M

11:31:59.586 INFO: ------------------------------------------------------------------------

11:31:59.586 ERROR: Error during SonarQube Scanner execution

java.lang.IllegalStateException: java.io.IOException: Cannot run program "/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" (in directory "/Users/runner/runners/2.159.2/work/1/s/arionstaffobjc"): error=2, No such file or directory

    at com.sonar.cpp.driver.ProcessExecutor.execute(ProcessExecutor.java:41)

    at com.sonar.cpp.analyzer.ClangDriver.lambda$probeCompiler$7(ClangDriver.java:499)

    at java.base/java.util.HashMap.computeIfAbsent(Unknown Source)

    at com.sonar.cpp.analyzer.ClangDriver.probeCompiler(ClangDriver.java:497)

    at com.sonar.cpp.analyzer.ClangDriver.onCapture(ClangDriver.java:320)

    at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:294)

    at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:206)

    at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:62)

    at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

    at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:410)

    at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:406)

    at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:369)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

    at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

    at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)

    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)

    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

    at java.base/java.lang.reflect.Method.invoke(Unknown Source)

    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)

    at com.sun.proxy.$Proxy0.execute(Unknown Source)

    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)

    at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:112)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:75)

    at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.io.IOException: Cannot run program "/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" (in directory "/Users/runner/runners/2.159.2/work/1/s/arionstaffobjc"): error=2, No such file or directory

    at java.base/java.lang.ProcessBuilder.start(Unknown Source)

    at java.base/java.lang.ProcessBuilder.start(Unknown Source)

    at com.sonar.cpp.driver.ProcessExecutor.execute(ProcessExecutor.java:30)

    ... 34 more

Caused by: java.io.IOException: error=2, No such file or directory

    at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)

    at java.base/java.lang.ProcessImpl.<init>(Unknown Source)

    at java.base/java.lang.ProcessImpl.start(Unknown Source)

mpaladin · October 31, 2019, 2:02pm

Hi @ivarorn,

is the following file present during sonar-scanner run?

/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang```

ivarorn · November 1, 2019, 10:03am

Yes clang is there.

I set up a task that calls clang with no parameters and i get a error from clang, but it seems not to be working when being called from sonar-scanner.

I now have set up a scanner for a swift project in Azure DevOps using the stock build task templates for my yaml file and that process worked flawlessly.

Setting up Sonar-Scanner for objc, c, c++ did not go as smooth. Is there no way of scanning a objc code in Azure DevOps without manually downloading the scanner, setting permissions and so on ?
Was hoping that I would be able to do that like with the Swift project ( setting up using the - task: SonarCloudPrepare@1 task, doing the XCode build using the - task: Xcode@5 task and finally doing the scan using - task: SonarCloudAnalyze@1 and publish using - task: SonarCloudPublish@1 ).

mpaladin · November 1, 2019, 2:39pm

Hi @ivarorn,

you can use the SonarCloud tasks, what you have to do manually is to download the build-wrapper and wrap the xcodebuild call which may not be possible using the Xcode@5 task (I am not an Azue DevOps expert).

ivarorn · November 5, 2019, 3:01pm

Ok… I have done exactly that now. Im downloading the custom wrapper, wrapping the build.
After that I execute the SonarCloudAnalyze@1 task that gives me the following error:
INFO: Load metrics repository (done) | time=43ms

##[error]WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$1 (file:/Users/runner/.sonar/cache/a89f1943fc75b65becd9fb4ecab8d913/sonar-tsql-plugin.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte,int,int,java.security.ProtectionDomain)

WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$1

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$1 (file:/Users/runner/.sonar/cache/a89f1943fc75b65becd9fb4ecab8d913/sonar-tsql-plugin.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte,int,int,java.security.ProtectionDomain)

WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$1

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

ERROR: Error during SonarQube Scanner execution

java.lang.IllegalStateException: java.io.EOFException: End of input at line 1 column 1

at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:311)

at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:206)

at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:62)

at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:410)

at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:406)

at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:369)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

3 more errors. Click on expand view in the context menu to view complete logs.

ivarorn · November 6, 2019, 11:14am

UPDATE:

I downloaded the build wrapper and scanner ( same versions ) to my local machine, and same command-lines work locally ( build, scan and upload to sonarcloud ).

So it seems that this has something to do with running in Azure Devops.

Are there any additional requirements that need to be fulfilled so that a scan can be done successfully in Azure DevOps ?

Azure DevOps image: macOS-10.14
Build wrapper version: build-wrapper-macosx-x86.zip
Sonar-Scanner version : sonar-scanner-cli-4.2.0.1873-macosx.zip

The main error seems to be that clang can not be found or executed.

10:59:10.518 ERROR: Error during SonarQube Scanner execution

java.lang.IllegalStateException: java.io.IOException: Cannot run program “/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang” (in directory “/Users/runner/runners/2.160.0/work/1/s/arionstaffobjc”): error=2, No such file or directory

at com.sonar.cpp.driver.ProcessExecutor.execute(ProcessExecutor.java:41)

at com.sonar.cpp.analyzer.ClangDriver.lambda$probeCompiler$7(ClangDriver.java:499)

at java.base/java.util.HashMap.computeIfAbsent(Unknown Source)

at com.sonar.cpp.analyzer.ClangDriver.probeCompiler(ClangDriver.java:497)

at com.sonar.cpp.analyzer.ClangDriver.onCapture(ClangDriver.java:320)

at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:294)

at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:206)

at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:62)

at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:410)

at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:406)

at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:369)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)

at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)

at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)

at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)

at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.base/java.lang.reflect.Method.invoke(Unknown Source)

at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)

at com.sun.proxy.$Proxy0.execute(Unknown Source)

at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)

at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)

at org.sonarsource.scanner.cli.Main.execute(Main.java:112)

at org.sonarsource.scanner.cli.Main.execute(Main.java:75)

at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.io.IOException: Cannot run program “/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang” (in directory “/Users/runner/runners/2.160.0/work/1/s/arionstaffobjc”): error=2, No such file or directory

If I list out files for XcodeDefault.xctoolchain/usr/bin directory i clearly see that clang is there
/Applications/Xcode_11.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I tried adding the same dir to PATH.
I also tried adding execute permissions to clang directly.

@mickaelcaro, @JolyLoic - Any ideas ?

mpaladin · November 11, 2019, 8:52am

Hi @ivarorn,

I tried to setup a reproducer project on azure devops but I am not able to reproduce the issue. Could you try to help me to reproduce it on a simple pipeline/project?

ivarorn · November 11, 2019, 8:58am

Is there any chance you could share the yaml file ? @mpaladin
Where should I send my source ?

mpaladin · November 11, 2019, 9:20am

Hi @ivarorn,

the one I checked is a small hello world project:

pool:
  name: Azure Pipelines
steps:
- bash: |
   set -eu
   
   xcodebuild -showsdks
   
   curl -L -O XXX/static/cpp/build-wrapper-macosx-x86.zip
   unzip build-wrapper-macosx-x86.zip
   
   ./build-wrapper-macosx-x86/build-wrapper-macosx-x86 --out-dir cfam-out xcodebuild build
   
   curl -L -O https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.2.0.1873-macosx.zip
   unzip sonar-scanner-cli-4.2.0.1873-macosx.zip
   ./sonar-scanner-4.2.0.1873-macosx/bin/sonar-scanner \
   -Dsonar.host.url=XXX \
   -Dsonar.login=XXX \
   -Dsonar.projectKey=test-massimo\
   -Dsonar.cfamily.build-wrapper-output=cfam-out \
   -Dsonar.verbose=true \
   -Dsonar.sources=.
  displayName: build

ivarorn · November 12, 2019, 10:07am

This works !

I was doing exactly the same thing, but with cmdline@2 azure devops pipeline tasks.
And I did the download and extract of sonar-scanner er build wrapper in powershell tasks.

Also by switching to curl and unzip in a bash task eliminated the need for setting file permissions with chmod command.

Thanks for your help

mpaladin · November 13, 2019, 9:40am

Hi @ivarorn,

indeed cmdline on macos might have been the cause of the issue braking something in the environment, better to use bash on macos.