GitLab CICD integration failed to set up GitLab Runner

zacktzeng · June 15, 2022, 2:53pm

Hello, I wrote a Terraform script to deploy SonarQube to GKE, create a network endpoint group, and configure the NEG to be one of the routes of GCP external load balancer. This load balancer has https set up.

Now when I tried to commit and merge the creation of .gitlab-ci.yml, I got this error message:

[0KRunning with gitlab-runner 14.4.2 (50fc80a6)e[0;m
e[0K  on gke-us-east1-gitlab-runner-6d7dbddd7d-tl54r G4B2n1ooe[0;m
section_start:1655303276:resolve_secrets
e[0Ke[0Ke[36;1mResolving secretse[0;me[0;m
section_end:1655303276:resolve_secrets
e[0Ksection_start:1655303276:prepare_executor
e[0Ke[0Ke[36;1mPreparing the "kubernetes" executore[0;me[0;m
e[0KUsing Kubernetes namespace: gitlab-runnere[0;m
e[0KUsing Kubernetes executor with image sonarsource/sonar-scanner-cli:latest ...e[0;m
e[0KUsing attach strategy to execute scripts...e[0;m
section_end:1655303276:prepare_executor
e[0Ksection_start:1655303276:prepare_script
e[0Ke[0Ke[36;1mPreparing environmente[0;me[0;m
Waiting for pod gitlab-runner/runner-g4b2n1oo-project-5207-concurrent-0h4nwd to be running, status is Pending
Running on runner-g4b2n1oo-project-5207-concurrent-0h4nwd via gke-us-east1-gitlab-runner-6d7dbddd7d-tl54r...

section_end:1655303280:prepare_script
e[0Ksection_start:1655303280:get_sources
e[0Ke[0Ke[36;1mGetting source from Git repositorye[0;me[0;m
e[32;1mFetching changes...e[0;m
Initialized empty Git repository in /builds/G4B2n1oo/0/security/sast-playground/sonarqube-playground-group/sonarqube-ce-test/.git/
e[32;1mCreated fresh repository.e[0;m
e[32;1mChecking out c7c4f858 as main...e[0;m

e[32;1mSkipping Git submodules setupe[0;m

section_end:1655303281:get_sources
e[0Ksection_start:1655303281:restore_cache
e[0Ke[0Ke[36;1mRestoring cachee[0;me[0;m
e[32;1mChecking cache for sonarqube-check...e[0;m
e[31;1mFATAL: file does not exist                        e[0;m 
e[0;33mFailed to extract cachee[0;m

section_end:1655303281:restore_cache
e[0Ksection_start:1655303281:step_script
e[0Ke[0Ke[36;1mExecuting "step_script" stage of the job scripte[0;me[0;m
e[32;1m$ sonar-scannere[0;m
INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /builds/G4B2n1oo/0/security/sast-playground/sonarqube-playground-group/sonarqube-ce-test/sonar-project.properties
INFO: SonarScanner 4.7.0.2747
INFO: Java 11.0.14 Alpine (64-bit)
INFO: Linux 5.4.170+ amd64
INFO: User cache: /builds/G4B2n1oo/0/security/sast-playground/sonarqube-playground-group/sonarqube-ce-test/.sonar/cache
ERROR: SonarQube server [https://sonarqube.internal.tulip.io] can not be reached
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 0.680s
INFO: Final Memory: 8M/887M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
org.sonarsource.scanner.api.internal.ScannerException: Unable to execute SonarScanner analysis
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:85)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:74)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:70)
	at org.sonarsource.scanner.api.EmbeddedScanner.doStart(EmbeddedScanner.java:185)
	at org.sonarsource.scanner.api.EmbeddedScanner.start(EmbeddedScanner.java:123)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:73)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Caused by: java.lang.IllegalStateException: Fail to get bootstrap index from server
	at org.sonarsource.scanner.api.internal.BootstrapIndexDownloader.getIndex(BootstrapIndexDownloader.java:42)
	at org.sonarsource.scanner.api.internal.JarDownloader.getScannerEngineFiles(JarDownloader.java:58)
	at org.sonarsource.scanner.api.internal.JarDownloader.download(JarDownloader.java:53)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:76)
	... 7 more
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:352)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:295)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:290)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1357)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1501)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1411)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:451)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:422)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.RealConnection.connectTls(RealConnection.java:336)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.RealConnection.establishProtocol(RealConnection.java:300)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.RealConnection.connect(RealConnection.java:185)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:224)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.ExchangeFinder.find(ExchangeFinder.java:88)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.Transmitter.newExchange(Transmitter.java:169)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
	at org.sonarsource.scanner.api.internal.shaded.okhttp.RealCall.execute(RealCall.java:81)
	at org.sonarsource.scanner.api.internal.ServerConnection.callUrl(ServerConnection.java:115)
	at org.sonarsource.scanner.api.internal.ServerConnection.downloadString(ServerConnection.java:99)
	at org.sonarsource.scanner.api.internal.BootstrapIndexDownloader.getIndex(BootstrapIndexDownloader.java:39)
	... 10 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	at java.base/sun.security.validator.Validator.validate(Validator.java:264)
	at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1341)
	... 44 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
	... 50 more
ERROR: 
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

section_end:1655303282:step_script
e[0Ksection_start:1655303282:cleanup_file_variables
e[0Ke[0Ke[36;1mCleaning up project directory and file based variablese[0;me[0;m

section_end:1655303283:cleanup_file_variables
e[0Ke[31;1mERROR: Job failed: command terminated with exit code 1
e[0;m

I noticed that there is a FATAL: file does not exist message in the terminal. I couldn’t identify what that file is though.

ganncamp · June 15, 2022, 6:52pm

Hi,

I can’t comment on your FATAL: file does not exist error. But your analysis error seems to be related to a certificate problem.

HTH,
Ann

zacktzeng · June 16, 2022, 1:48pm

I see. Is there a possibility to deploy a SonarQube behind a GCP global load balancer?

ganncamp · June 16, 2022, 1:52pm

Hi,

I don’t see why you would need to.

Isn’t the point of a load balancer to split the incoming traffic among multiple, equal nodes? There’s only going to be one SonarQube node…

Ann

zacktzeng · June 16, 2022, 1:57pm

Hi, our load balancer implements https for the services behind. It’s more for the https than for load balancing.

zacktzeng · June 16, 2022, 1:58pm

If I want to implement https for SonarQube without hooking it to an existing load balancer, is it true that I will need to configure the ingress? I’m using helm chart.

Sylvain_Combe · June 16, 2022, 2:47pm

Hi @zacktzeng
as Ann mentioned your issue at hand is about HTTPS/SSL.
The scanner must be equipped with the right certificates to trust your HTTPS front-end, regardless of its nature.
You’ll find some inputs about this topic in the Advanced Docker Configuration section of the SonarScanner documentation page.
Sylvain

zacktzeng · June 16, 2022, 3:54pm

So basically it’s the sonar scanner that needs modification, not sonarqube. The “Advanced Docker Configuration” section talks about including the modified cacerts file and create a new docker image with the cert. If I’m using the scanner in GitLab’s .gitlab-ci.yml that is essentially what I need to do right? I just need to change the name of the image inside .gitlab-ci.yml.

sonarqube-check:
  image: 
    name: <my-own-source>/sonar-scanner-cli:latest
    entrypoint: [""]
  variables:
    SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar"  # Defines the location of the analysis task cache
    GIT_DEPTH: "0"  # Tells git to fetch all the branches of the project, required by the analysis task
  cache:
    key: "${CI_JOB_NAME}"
    paths:
      - .sonar/cache
  script: 
    - sonar-scanner
  allow_failure: true
  only:
    - merge_requests
    - main # or the name of your main branch
    - develop

Sylvain_Combe · June 17, 2022, 7:51am

Hi Zack
yes, using your own image, equipped with your private trust store file seems the best approach.

GitLab docker executors don’t support volume mounts yet. Other alternatives would therefore require you to script an update of the sonar-scanner-cli certificates store from within the pipeline definition file.