Sonar-Job in gitlab pipeline somestime crash with java.nio.file.NoSuchFileException

cavit · February 10, 2022, 12:19pm

Hello together,

we have a sonar-scan-job in our gitlab pipeline which sometimes crashes. I would say every twentieth job crashes with the java.nio.file.NoSuchFileException. When i retry the job 3-4 times it will somehow work.

Command: sonar-scanner -Dsonar.qualitygate.wait=true -Dsonar.login=$SONAR_LOGIN
Versions

SonarScanner 4.6.2.2472
Java 11.0.11 Alpine (64-bit)
Linux 4.14.231-173.361.amzn2.x86_64 amd64

The error:

08:51:36.868 DEBUG: Adding rules for repository 'jssecurity', language: JAVASCRIPT, [class com.sonar.security.frontend.js.rules.DomOpenRedirectCheck, class com.sonar.security.frontend.js.rules.DomXssCheck, class com.sonar.security.frontend.js.rules.DynamicCodeExecutionCheck, class com.sonar.security.frontend.js.rules.FilePathInjectionCheck, class com.sonar.security.frontend.js.rules.HttpRedirectionCheck, class com.sonar.security.frontend.js.rules.NoSQLInjectionCheck, class com.sonar.security.frontend.js.rules.OSArgumentConstructionCheck, class com.sonar.security.frontend.js.rules.OSArgumentInjectionCheck, class com.sonar.security.frontend.js.rules.OSCommandInjectionCheck, class com.sonar.security.frontend.js.rules.ReflectedXssCheck, class com.sonar.security.frontend.js.rules.RegularExpressionDoSCheck, class com.sonar.security.frontend.js.rules.ServerSideRequestCheck, class com.sonar.security.frontend.js.rules.SQLInjectionCheck, class com.sonar.security.frontend.js.rules.ZipSlipCheck, class com.sonar.security.frontend.js.rules.SessionFixationCheck] from com.sonar.security.frontend.js.D
08:51:36.878 DEBUG: Adding rules for repository 'tssecurity', language: TYPESCRIPT, [class com.sonar.security.frontend.js.rules.DomOpenRedirectCheck, class com.sonar.security.frontend.js.rules.DomXssCheck, class com.sonar.security.frontend.js.rules.DynamicCodeExecutionCheck, class com.sonar.security.frontend.js.rules.FilePathInjectionCheck, class com.sonar.security.frontend.js.rules.HttpRedirectionCheck, class com.sonar.security.frontend.js.rules.NoSQLInjectionCheck, class com.sonar.security.frontend.js.rules.OSArgumentConstructionCheck, class com.sonar.security.frontend.js.rules.OSArgumentInjectionCheck, class com.sonar.security.frontend.js.rules.OSCommandInjectionCheck, class com.sonar.security.frontend.js.rules.ReflectedXssCheck, class com.sonar.security.frontend.js.rules.RegularExpressionDoSCheck, class com.sonar.security.frontend.js.rules.ServerSideRequestCheck, class com.sonar.security.frontend.js.rules.SQLInjectionCheck, class com.sonar.security.frontend.js.rules.ZipSlipCheck, class com.sonar.security.frontend.js.rules.SessionFixationCheck] from com.sonar.security.frontend.js.B
08:51:37.195 DEBUG: 'Import external issues report' skipped because one of the required properties is missing
08:51:37.197 DEBUG: 'IaC Terraform Sensor' skipped because there is no related file in current project
08:51:37.198 DEBUG: 'PL/SQL Sensor' skipped because there is no related file in current project
08:51:37.198 DEBUG: 'Scala Sensor' skipped because there is no related file in current project
08:51:37.199 DEBUG: 'Scoverage sensor for Scala coverage' skipped because there is no related file in current project
08:51:37.199 DEBUG: 'Import of Scalastyle issues' skipped because there is no related file in current project
08:51:37.200 DEBUG: 'Import of Scapegoat issues' skipped because there is no related file in current project
08:51:37.200 DEBUG: 'Import of Checkstyle issues' skipped because one of the required properties is missing
08:51:37.200 DEBUG: 'Import of PMD issues' skipped because one of the required properties is missing
08:51:37.201 DEBUG: 'Import of SpotBugs issues' skipped because one of the required properties is missing
08:51:37.201 DEBUG: 'Removed properties sensor' skipped because one of the required properties is missing
08:51:37.202 DEBUG: 'Flex' skipped because there is no related file in current project
08:51:37.202 DEBUG: 'Flex Cobertura' skipped because there is no related file in current project
08:51:37.203 DEBUG: 'Swift Code Quality and Security' skipped because there is no related file in current project
08:51:37.203 DEBUG: 'Import of SwiftLint issues' skipped because there is no related file in current project
08:51:37.204 DEBUG: 'gcov' skipped because there is no related file in current project
08:51:37.205 DEBUG: 'llvm-cov' skipped because there is no related file in current project
08:51:37.205 DEBUG: 'cppunit' skipped because there is no related file in current project
08:51:37.206 DEBUG: 'VisualStudioCoverage' skipped because there is no related file in current project
08:51:37.207 DEBUG: 'bullseye' skipped because there is no related file in current project
08:51:37.207 DEBUG: 'Python Sensor' skipped because there is no related file in current project
08:51:37.207 DEBUG: 'Cobertura Sensor for Python coverage' skipped because there is no related file in current project
08:51:37.207 DEBUG: 'PythonXUnitSensor' skipped because there is no related file in current project
08:51:37.208 DEBUG: 'Import of Pylint issues' skipped because there is no related file in current project
08:51:37.208 DEBUG: 'Import of Bandit issues' skipped because there is no related file in current project
08:51:37.208 DEBUG: 'Import of Flake8 issues' skipped because there is no related file in current project
08:51:37.209 DEBUG: 'Code Quality and Security for Go' skipped because there is no related file in current project
08:51:37.209 DEBUG: 'Go Unit Test Report' skipped because there is no related file in current project
08:51:37.209 DEBUG: 'Go Cover sensor for Go coverage' skipped because one of the required properties is missing
08:51:37.210 DEBUG: 'Import of go vet issues' skipped because there is no related file in current project
08:51:37.211 DEBUG: 'Import of Golint issues' skipped because there is no related file in current project
08:51:37.211 DEBUG: 'Import of GoMetaLinter issues' skipped because there is no related file in current project
08:51:37.211 DEBUG: 'Import of GolangCI-Lint issues' skipped because there is no related file in current project
08:51:37.211 DEBUG: 'Kotlin Sensor' skipped because there is no related file in current project
08:51:37.212 DEBUG: 'KotlinSurefireSensor' skipped because there is no related file in current project
08:51:37.212 DEBUG: 'Import of detekt issues' skipped because there is no related file in current project
08:51:37.213 DEBUG: 'Import of Android Lint issues' skipped because one of the required properties is missing
08:51:37.213 DEBUG: 'Import of ktlint issues' skipped because there is no related file in current project
08:51:37.213 DEBUG: 'T-SQL Sensor' skipped because there is no related file in current project
08:51:37.214 DEBUG: 'Import of ESLint issues' skipped because one of the required properties is missing
08:51:37.214 DEBUG: 'Import of TSLint issues' skipped because one of the required properties is missing
08:51:37.214 DEBUG: 'CSS Metrics' skipped because there is no related file in current project
08:51:37.215 DEBUG: 'Import of stylelint issues' skipped because there is no related file in current project
08:51:37.215 DEBUG: 'Ruby Sensor' skipped because there is no related file in current project
08:51:37.216 DEBUG: 'Import of RuboCop issues' skipped because there is no related file in current project
08:51:37.216 DEBUG: 'SimpleCov Sensor for Ruby coverage' skipped because there is no related file in current project
08:51:37.216 DEBUG: 'Python HTML templates processing' skipped because there is no related file in current project
08:51:37.217 DEBUG: 'PHP sensor' skipped because there is no related file in current project
08:51:37.217 DEBUG: 'Analyzer for "php.ini" files' skipped because there is no related file in current project
08:51:37.218 DEBUG: 'Import of PHPStan issues' skipped because there is no related file in current project
08:51:37.218 DEBUG: 'Import of Psalm issues' skipped because there is no related file in current project
08:51:37.218 DEBUG: 'AbapSquidSensor' skipped because there is no related file in current project
08:51:37.224 DEBUG: 'CFamily' skipped because there is no related file in current project
08:51:37.225 DEBUG: Sensors : JavaSensor -> IaC CloudFormation Sensor -> C# Project Type Information -> C# Analysis Log -> C# Properties -> SurefireSensor -> JavaXmlSensor -> HTML -> XML Sensor -> VB.NET Project Type Information -> VB.NET Analysis Log -> VB.NET Properties -> JaCoCo XML Report Importer -> JavaScript analysis -> TypeScript analysis -> JavaScript/TypeScript Coverage -> CSS Rules -> ThymeLeaf template sensor -> Serverless configuration file sensor -> AWS SAM template file sensor -> Generic Test Executions Report -> JavaSecuritySensor -> CSharpSecuritySensor -> PhpSecuritySensor -> PythonSecuritySensor -> JsSecuritySensor
08:51:37.229 INFO: Sensor JavaSensor [java]
08:51:37.539 INFO: Configured Java source version (sonar.java.source): 11
08:51:37.545 INFO: JavaClasspath initialization
08:51:37.719 INFO: ------------------------------------------------------------------------
08:51:37.719 INFO: EXECUTION FAILURE
08:51:37.719 INFO: ------------------------------------------------------------------------
08:51:37.719 INFO: Total time: 9.865s
08:51:37.758 INFO: Final Memory: 24M/94M
08:51:37.758 INFO: ------------------------------------------------------------------------
08:51:37.758 ERROR: Error during SonarScanner execution
java.lang.IllegalStateException: java.nio.file.NoSuchFileException: /builds/product-team/product/.git/.probe-4d302a54-1f04-47be-b561-d9e068bd85c8
	at org.sonar.java.classpath.AbstractClasspath.getFilesInDir(AbstractClasspath.java:190)
	at org.sonar.java.classpath.AbstractClasspath.getFilesForPattern(AbstractClasspath.java:176)
	at org.sonar.java.classpath.AbstractClasspath.getFilesFromProperty(AbstractClasspath.java:126)
	at org.sonar.java.classpath.ClasspathForMain.init(ClasspathForMain.java:58)
	at org.sonar.java.classpath.AbstractClasspath.getElements(AbstractClasspath.java:321)
	at org.sonar.java.SonarComponents.getJavaClasspath(SonarComponents.java:183)
	at org.sonar.java.JavaFrontend.<init>(JavaFrontend.java:89)
	at org.sonar.plugins.java.JavaSensor.execute(JavaSensor.java:108)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:79)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:384)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:380)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:349)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Caused by: java.nio.file.NoSuchFileException: /builds/product-team/product/.git/.probe-4d302a54-1f04-47be-b561-d9e068bd85c8
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
	at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2761)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2797)
	at org.sonar.java.classpath.AbstractClasspath$AbstractFileFinder.find(AbstractClasspath.java:284)
	at org.sonar.java.classpath.AbstractClasspath.getMatchingDirs(AbstractClasspath.java:215)
	at org.sonar.java.classpath.AbstractClasspath.getFilesInDir(AbstractClasspath.java:187)
	... 37 more
08:51:37.760 ERROR: 
08:51:37.760 ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

Do you have any hint for me, why this sometimes happens? A .probe-4d302a54-1f04-47be-b561-d9e068bd85c8 does not exist in the repository.

Julien_HENRY · February 10, 2022, 7:34pm

Hi,

Seems you are trying to analyze a Java project, and the classpath property is not set correctly, since it is somehow trying to find classes in the .git folder.
It is common on some CI to have temporary files created for a brief moment by background process, so this error simply mean that during the listing of the directory, a file was there during a brief moment, and it creates an inconsistency.

If you are analyzing a Java project, why not using the Scanner for Maven or the Scanner for Gradle? They will take care of computing the Java classpath analysis property for you.

If you want to stick to the SonarScanner with manual configuration, avoid things like:

sonar.java.binaries=.
sonar.java.libraries=**/*

since it will probably include too much.

If you need more help, please share you sonar-project.properties file.

cavit · February 11, 2022, 5:34am

Good morning,

thank you for the fast answer.

This is our current sonar-project.properties

# The project key that is unique for each project.
# Allowed characters are: letters, numbers, '-', '_', '.' and ':', with at least one non-digit.
# When using Maven, it is automatically set to <groupId>:<artifactId>.
sonar.projectKey=product
# sonar.sources: no pattern support, add each src folder
sonar.sources=\
  app/core/src/main/java,\
  app/ui/ui-backend/src/main,\
  app/ui/ui-frontend/src,\
  app/ui/traceparts/src/main/java
sonar.tests=\
  app/core/src/test/java,\
  app/ui/ui-backend/src/test/java,\
  app/ui/ui-frontend/tests/unit,\
  app/ui/traceparts/src/test/java
sonar.java.binaries=\
  **/target/classes,\
  **/target/fat-jar/BOOT-INF/classes
sonar.java.libraries=**/target/fat-jar/BOOT-INF/lib/*.jar
sonar.javascript.lcov.reportPaths=app/ui/ui-frontend/reports/coverage/lcov.info
sonar.coverage.jacoco.xmlReportPaths=\
  app/core/target/site/jacoco/jacoco.xml,\
  app/ui/ui-backend/target/site/jacoco/jacoco.xml, \
  app/ui/traceparts/target/site/jacoco/jacoco.xml
sonar.qualitygate.wait=true
sonar.exclusions=\
  **/crisp/product/cfg/WebSecurityConfig.java,\
  **/crisp/product/cfg/WebViews.java,\
  **/crisp/product/core/Profiles.java,\
  **/crisp/product/core/WebPaths.java, \
  app/ui/ui-frontend/src/setup/vue.js, \
  app/ui/ui-frontend/src/lang/messages.js, \
  app/ui/ui-frontend/src/bundles/**/Bundle.vue, \
  app/ui/ui-frontend/src/bundles/**/main.ts, \
  app/ui/ui-frontend/src/bundles/**/main.js, \
# add for sonar unit-test-results
sonar.junit.reportPaths=\
  app/core/target/surefire-reports,\
  app/ui/ui-backend/target/surefire-reports,\
  app/ui/traceparts/target/surefire-reports
sonar.testExecutionReportPaths=app/ui/ui-frontend/reports/reports-for-sonar/test-report.xml
sonar.java.source=11
sonar.verbose=true

any recommandation to fix this problem?

Julien_HENRY · February 11, 2022, 9:00am

Because your pattern starts with ** then the entire project filesystem will be scanned for matches.

I will report the issue on our side since I think we could be more resilient.

In the meantime, you should avoid using patterns starting by **. Instead, you should use relative paths like:

sonar.java.binaries=\
  app/core/target/classes,\
  app/ui/ui-backend/target/classes,\
  app/ui/ui-frontend/target/classes,\
  [...]

But it seems you are analyzing a Maven project, so the easiest solution is probably to use the Scanner for Maven.

cavit · February 11, 2022, 12:20pm

Ok, thank you for you fast help!

Regards, Cavit

system · February 18, 2022, 12:20pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.