lawrence
January 30, 2020, 3:58pm
1
SonarQube: 8.0-developer Edition
I’m trying to run a scan on my code, and am getting the following error:
ERROR: Error during SonarQube Scanner execution
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: build/linux/x86_64/_deps/boost-src/libs/wave/test/testwave/testfiles/utf8-test-??????????
Scanner command is:
build-wrapper-linux-x86-64 --out-dir sonar-dir make -C build/linux/x86_64 all test
sonar-scanner \
-Dsonar.login=$SONAR_TOKEN \
-Dsonar.projectKey=$projectKey\
-Dsonar.sources=. \
-Dsonar.cfamily.build-wrapper-output=sonar-dir \
-Dsonar.exclusions="build/linux/x86_64/_deps/**/*"
What’s weird is that I’ve added the sonar.exclusions in, however it is still trying to execute the scan on that file?
Am I missing something?
Thanks in advance!
mpaladin
January 31, 2020, 8:14am
2
Hi @lawrence ,
could you please share the entire log and error stacktrace?
lawrence
January 31, 2020, 9:42am
3
Hi @mpaladin ,
Here is the logs:
INFO: Scanner configuration file: /usr/lib/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: NONE
INFO: SonarQube Scanner 4.2.0.1873
INFO: Java 11.0.3 AdoptOpenJDK (64-bit)
INFO: Linux 5.3.0-29-generic amd64
INFO: User cache: /root/.sonar/cache
INFO: SonarQube server 8.0.0
INFO: Default locale: "en_US", source code encoding: "US-ASCII" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=147ms
INFO: Server id: <redacted>
INFO: User cache: /root/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=50ms
INFO: Load/download plugins (done) | time=6781ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Process project properties (done) | time=6ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=4ms
INFO: Project key: <projectkey>
INFO: Base dir: /builds/
INFO: Working dir: /builds/.scannerwork
INFO: Load project settings for component key: '<projectkey>'
INFO: Load project settings for component key: '<projectkey>' (done) | time=24ms
INFO: Load project branches
INFO: Load project branches (done) | time=34ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=16ms
INFO: Load branch configuration
INFO: Detected 'GitLab'
INFO: Auto-configuring branch 'add-sonarqube'
INFO: Load branch configuration (done) | time=3ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=58ms
INFO: Detected Gitlab CI
INFO: Load active rules
INFO: Load active rules (done) | time=876ms
INFO: Branch name: add-sonarqube, type: short living
INFO: SCM collecting changed files in the branch
INFO: SCM collecting changed files in the branch (done) | time=214ms
INFO: Indexing files...
INFO: Project configuration:
INFO: Excluded sources: build/linux/x86_64/_deps/**/*
INFO: Excluded sources for coverage: **/tests/*, *.test.*
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 11.976s
INFO: Final Memory: 15M/80M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarQube Scanner execution
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: build/linux/x86_64/_deps/boost-src/libs/wave/test/testwave/testfiles/utf8-test-??????????
at java.base/sun.nio.fs.UnixPath.encode(Unknown Source)
at java.base/sun.nio.fs.UnixPath.<init>(Unknown Source)
at java.base/sun.nio.fs.UnixFileSystem.getPath(Unknown Source)
at java.base/java.nio.file.Path.resolve(Unknown Source)
at org.sonarsource.scm.git.IncludedFilesRepository.indexFiles(IncludedFilesRepository.java:65)
at org.sonarsource.scm.git.IncludedFilesRepository.<init>(IncludedFilesRepository.java:40)
at org.sonarsource.scm.git.GitIgnoreCommand.init(GitIgnoreCommand.java:37)
at org.sonar.scanner.scan.filesystem.ProjectFileIndexer.index(ProjectFileIndexer.java:104)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:341)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
at org.sonarsource.scanner.cli.Main.main(Main.java:61)
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
How do I generate the error stacktrace?
Thanks
Hi,
The issue come from this file whose name seems to contain some fancy characters:
The issue is likely in JGit, but to be sure could you please share the raw filename (as you can see in the logs the unmappable characters have been replaced by ?).
lawrence
January 31, 2020, 11:49am
6
Hi @Julien_HENRY ,
Yup, the folder name is utf8-test-ßµ™∃.
However I thought that would be fine to be excluded in sonar.exclusions?
How would you suggest I should proceed?
Thanks
After more tests, everything works fine on my side, even with a file/folder named like your. I was able to reproduce by first running:
export LANG=C
(before the value was fr_FR.UTF-8)https://stackoverflow.com/a/39843086/534773 )
I suggest you check your environment variables LC_* / LANG values, and possibly change them to en_US.UTF-8 before running the SQ analysis.
2 Likes
lawrence
January 31, 2020, 3:13pm
8
Hi @Julien_HENRY ,
Thank you for looking into this, that was the issue!
Was required to do the following to fix it:
- apt-get update && apt-get install -y locales
- locale-gen "en_US.UTF-8"
- update-locale LC_ALL="en_US.UTF-8"
- export LANG="en_US.UTF-8"
Out of curiosity, should it not have been ignored by sonar.exclusions?
Thanks
2 Likes
Possibly, but that is not easy to do here. To compare the SCM file path to the sonar.exclusions pattern we have to “resolve” the path anyway, and this is the operation that was failing.
2 Likes
lawrence
January 31, 2020, 4:06pm
10
Gotcha, thanks for the explanation (and help
)!