Oops! I should probably also mention that:
- Our code is Dockerized
- Everything we do is on Linux containers
I tried adding sonar.visualstudio.enable=true to my sonar-project.properties file, but that seemed to have zero effect.
Here’s an example of my latest log output:
#!bash
set -e
VERSION=4.1.0.1829
SONAR_TOKEN=$SONAR_TOKEN
SCANNER_DIRECTORY=/tmp/cache/scanner
export SONAR_USER_HOME=$SCANNER_DIRECTORY/.sonar
OS="linux"
echo $SONAR_USER_HOME
if [[ ! -x "$SCANNER_DIRECTORY/sonar-scanner-$VERSION-$OS/bin/sonar-scanner" ]]; then
curl -Ol https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-$VERSION-$OS.zip
unzip -qq -o sonar-scanner-cli-$VERSION-$OS.zip -d $SCANNER_DIRECTORY
fi
chmod +x $SCANNER_DIRECTORY/sonar-scanner-$VERSION-$OS/bin/sonar-scanner
chmod +x $SCANNER_DIRECTORY/sonar-scanner-$VERSION-$OS/jre/bin/java
$SCANNER_DIRECTORY/sonar-scanner-$VERSION-$OS/bin/sonar-scanner
Loading local .env
RL_DOCKER_REPO_BASE=<<REDACTED>>
RL_APP_NAME=kr-authorization
RL_APP_TAG=latest
RL_APP_TEST_CS_TAG=latest-test-cs
RL_APP_TEST_SPA_UNIT_TAG=latest-test-spa-unit
RL_APP_PORT=5401
NPM_REGISTRY=registry.npmjs.org
TZ=America/Chicago
ASPNETCORE_URLS=https://+:5001
ASPNETCORE_ENVIRONMENT=docker-local
ASPNETCORE_Kestrel__Certificates__Default__Path=/certs/redline.pfx
ASPNETCORE_Kestrel__Certificates__Default__Password=password
Authentication__SigningCert__Path=/certs/redline.pfx
Authentication__SigningCert__Password=password
Authentication__DataProtectionCert__Path=/certs/redline.pfx
Authentication__DataProtectionCert__Password=password
A3__Authentication__PublicKeyPath=/certs/redline-public.crt
TOKEN_SIGNING_PUBLIC_KEY_PATH=/certs/redline-public.crt
POSTGRES_HOST=pgsqldb
POSTGRES_PORT=5432
POSTGRES_USER=postgres
POSTGRES_PASSWORD=<<REDACTED>>
RABBITMQ_HOST=rabbitmq
RABBITMQ_USER=admin
RABBITMQ_PASSWORD=<<REDACTED>>
AWS_REGION=*********
Loaded local .env...
Overriding default values with build scripts...
RL_APP_NAME -> kr-authorization
RL_IMAGE_BASE -> <<REDACTED>>/kr-authorization
ASPNETCORE_ENVIRONMENT -> docker-********
SAFE_BRANCH -> pull_258
RL_SHA_SHORT -> 6c5968c
RL_APP_TAG -> 6c5968c-pull_258-pr
RL_APP_TAG_LATEST -> latest-pull_258-pr
RL_APP_TEST_CS_TAG -> 6c5968c-pull_258-pr-test-cs
RL_APP_TEST_SPA_UNIT_TAG -> 6c5968c-pull_258-pr-test-spa-unit
RL_APP_TEST_SPA_INTEGRATION_TAG -> 6c5968c-pull_258-pr-test-spa-integration
IS_PR_BUILD -> "true"
IS_BRANCH_BUILD -> "false"
IS_MASTER_BUILD -> "false"
Docker volume environment variables...
RL_DOCKER_VOLS_ROOT -> /home/********/docker-volumes
RL_DOCKER_VOLS_IN -> /home/********/docker-volumes/in
RL_DOCKER_VOLS_OUT -> /home/********/docker-volumes/out
Docker image environment variables...
RL_IMAGE_BASE -> <<REDACTED>>/kr-authorization
RL_IMAGE -> <<REDACTED>>/kr-authorization:6c5968c-pull_258-pr
RL_IMAGE_LATEST -> <<REDACTED>>/kr-authorization:latest-pull_258-pr
RL_IMAGE_TEST_CS -> <<REDACTED>>/kr-authorization:6c5968c-pull_258-pr-test-cs
RL_IMAGE_TEST_SPA_UNIT -> <<REDACTED>>/kr-authorization:6c5968c-pull_258-pr-test-cs
RL_IMAGE_TEST_SPA_INTEGRATION -> <<REDACTED>>/kr-authorization:6c5968c-pull_258-pr-test-spa-integration
Dns name that containers use to communicate with the docker host running them...
RL_DOCKER_HOST -> host.docker.internal
baseUrl -> https://localhost:5001
(end of loading environment)
/tmp/cache/scanner/.sonar
INFO: Scanner configuration file: /tmp/cache/scanner/sonar-scanner-4.1.0.1829-linux/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/********/app/sonar-project.properties
INFO: SonarQube Scanner 4.1.0.1829
INFO: Java 11.0.3 AdoptOpenJDK (64-bit)
INFO: Linux 4.15.0-1027-gcp amd64
INFO: User cache: /tmp/cache/scanner/.sonar/cache
INFO: SonarQube server 8.0.0
INFO: Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=667ms
INFO: Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
INFO: User cache: /tmp/cache/scanner/.sonar/cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=159ms
INFO: Load/download plugins (done) | time=533ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Execute project builders
INFO: Execute project builders (done) | time=3ms
INFO: Project key: <<REDACTED>>_Krypton.Authorization
INFO: Base dir: /home/********/app
INFO: Working dir: /home/********/app/.scannerwork
INFO: Load project settings for component key: '<<REDACTED>>_Krypton.Authorization'
INFO: Load project settings for component key: '<<REDACTED>>_Krypton.Authorization' (done) | time=221ms
INFO: Found an active CI vendor: 'CircleCI'
INFO: Load project branches
INFO: Load project branches (done) | time=126ms
INFO: Check ALM binding of project '<<REDACTED>>_Krypton.Authorization'
INFO: Detected project binding: BOUND
INFO: Check ALM binding of project '<<REDACTED>>_Krypton.Authorization' (done) | time=112ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=122ms
INFO: Load branch configuration
INFO: Auto-configuring pull request 258
INFO: Load branch configuration (done) | time=701ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=167ms
INFO: Load active rules
INFO: Load active rules (done) | time=3782ms
INFO: Organization key: <<REDACTED>>
INFO: Pull request 258 for merge into master from master
INFO: SCM collecting changed files in the branch
INFO: SCM collecting changed files in the branch (done) | time=590ms
INFO: Indexing files...
INFO: Project configuration:
INFO: 457 files indexed
INFO: 0 files ignored because of scm ignore settings
INFO: Quality profile for cs: Sonar way
INFO: Quality profile for css: Sonar way
INFO: Quality profile for js: Sonar way
INFO: Quality profile for plsql: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: Quality profile for web: Sonar way
INFO: ------------- Run sensors on module <<REDACTED>>_Krypton.Authorization
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=127ms
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$1 (file:/tmp/cache/scanner/.sonar/cache/a89f1943fc75b65becd9fb4ecab8d913/sonar-tsql-plugin.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO: Sensor SonarCSS Metrics [cssfamily]
INFO: Sensor SonarCSS Metrics [cssfamily] (done) | time=185ms
INFO: Sensor SonarCSS Rules [cssfamily]
ERROR: CSS rules were not executed. Only Node.js v8 or later is supported, got v6.1.0.
org.sonarsource.nodejs.NodeCommandException: Only Node.js v8 or later is supported, got v6.1.0.
at org.sonarsource.nodejs.NodeCommandBuilderImpl.checkNodeCompatibility(NodeCommandBuilderImpl.java:172)
at org.sonarsource.nodejs.NodeCommandBuilderImpl.build(NodeCommandBuilderImpl.java:144)
at org.sonar.css.plugin.server.CssAnalyzerBridgeServer.initNodeCommand(CssAnalyzerBridgeServer.java:133)
at org.sonar.css.plugin.server.CssAnalyzerBridgeServer.startServer(CssAnalyzerBridgeServer.java:101)
at org.sonar.css.plugin.server.CssAnalyzerBridgeServer.startServerLazily(CssAnalyzerBridgeServer.java:153)
at org.sonar.css.plugin.CssRuleSensor.execute(CssRuleSensor.java:99)
at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:45)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:48)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:66)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:48)
at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:68)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:122)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:108)
at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:421)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:417)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:375)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:122)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:108)
at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:126)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:122)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:108)
at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:58)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:52)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:185)
at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:137)
at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
at org.sonarsource.scanner.cli.Main.main(Main.java:61)
INFO: Sensor SonarCSS Rules [cssfamily] (done) | time=828ms
INFO: Sensor PL/SQL Sensor [plsql]
WARN: The Data Dictionary is not configured which prevents rule(s) S3641, S3921, S3618, S3651 to raise issues. See https://docs.sonarqube.org/display/PLUG/Data+Dictionary
INFO: 17 source files to be analyzed
INFO: 17/17 source files have been analyzed
INFO: Sensor PL/SQL Sensor [plsql] (done) | time=870ms
INFO: Sensor C# Properties [csharp]
WARN: Property missing: 'sonar.cs.analyzer.projectOutPaths'. No protobuf files will be loaded for this project.
WARN: No Roslyn issues report found for this project.
INFO: Sensor C# Properties [csharp] (done) | time=2ms
INFO: Sensor JavaXmlSensor [java]
INFO: Sensor JavaXmlSensor [java] (done) | time=14ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=150ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=9ms
INFO: Sensor JavaScript analysis [javascript]
INFO: TypeScript dependency was not found inside project directory, Node.js will search TypeScript using module resolution algorithm; analysis will fail without TypeScript.
ERROR: Only Node.js v8 or later is supported, got v6.1.0.
org.sonarsource.nodejs.NodeCommandException: Only Node.js v8 or later is supported, got v6.1.0.
at org.sonarsource.nodejs.NodeCommandBuilderImpl.checkNodeCompatibility(NodeCommandBuilderImpl.java:172)
at org.sonarsource.nodejs.NodeCommandBuilderImpl.build(NodeCommandBuilderImpl.java:144)
at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.initNodeCommand(EslintBridgeServerImpl.java:148)
at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.startServer(EslintBridgeServerImpl.java:106)
at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.startServerLazily(EslintBridgeServerImpl.java:169)
at org.sonar.plugins.javascript.eslint.AbstractEslintSensor.execute(AbstractEslintSensor.java:106)
at org.sonar.plugins.javascript.eslint.JavaScriptEslintBasedSensor.execute(JavaScriptEslintBasedSensor.java:49)
at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:45)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:48)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:66)
at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:48)
at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:68)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:122)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:108)
at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:421)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:417)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:375)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:122)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:108)
at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:126)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:122)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:108)
at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:58)
at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:52)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:185)
at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:137)
at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
at org.sonarsource.scanner.cli.Main.main(Main.java:61)
INFO: Sensor JavaScript analysis [javascript] (done) | time=1902ms
INFO: Sensor TypeScript analysis [javascript]
INFO: Sensor TypeScript analysis [javascript] (done) | time=0ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=5ms
INFO: Sensor JavaSecuritySensor [security]
INFO: Reading type hierarchy from: /home/********/app/.scannerwork/ucfg2/java
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /home/********/app/.scannerwork/ucfg2/java
INFO: No UCFGs have been included for analysis.
INFO: Sensor JavaSecuritySensor [security] (done) | time=10ms
INFO: Sensor CSharpSecuritySensor [security]
INFO: Reading type hierarchy from: /home/********/app/ucfg_cs2
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /home/********/app/ucfg_cs2
INFO: No UCFGs have been included for analysis.
INFO: Sensor CSharpSecuritySensor [security] (done) | time=5ms
INFO: Sensor PhpSecuritySensor [security]
INFO: Reading type hierarchy from: /home/********/app/.scannerwork/ucfg2/php
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /home/********/app/.scannerwork/ucfg2/php
INFO: No UCFGs have been included for analysis.
INFO: Sensor PhpSecuritySensor [security] (done) | time=7ms
INFO: Sensor PythonSecuritySensor [security]
INFO: Reading type hierarchy from: /home/********/app/.scannerwork/ucfg2/python
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /home/********/app/.scannerwork/ucfg2/python
INFO: No UCFGs have been included for analysis.
INFO: Sensor PythonSecuritySensor [security] (done) | time=1ms
INFO: ------------- Run sensors on project
INFO: Sensor C# [csharp]
INFO: Sensor C# [csharp] (done) | time=2ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=20ms
INFO: CPD Executor 5 files had no CPD blocks
INFO: CPD Executor Calculating CPD for 22 files
INFO: CPD Executor CPD calculation finished (done) | time=59ms
INFO: SCM writing changed lines
INFO: SCM writing changed lines (done) | time=31ms
INFO: Analysis report generated in 368ms, dir size=204 KB
INFO: Analysis report compressed in 339ms, zip size=127 KB
INFO: Analysis report uploaded in 667ms
INFO: ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=<<REDACTED>>_Krypton.Authorization&pullRequest=258
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=AXRhXWZS9WAKvHtxVvJ-
INFO: Analysis total time: 20.118 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 24.187s
INFO: Final Memory: 28M/97M
INFO: ------------------------------------------------------------------------
CircleCI received exit code 0
So it seems to be finding my source code okay and running the scan fine. From reading, it also seems the ugly NodeCommandException is expected and not something I should worry about. The INFO: Sensor C# [csharp] (done) | time=2ms line - as well as some similar ones - tells me that Sonar is basically not doing anything regarding C#.
Anyway, I’m gonna keep trying, but if anyone has ideas I’m all ears!
Actually, here’s my CircleCI configuration:
# CircleCI file for CI/CD integration
version: 2.1
workflows:
build_workflow:
jobs:
- build:
context: default-context
docker-image-repo: <<REDACTED>>
run-tests-csharp: true
run-tests-spa-unit: true
run-tests-spa-integration: false
orbs:
sonarcloud: sonarsource/sonarcloud@1.0.1
jobs:
build:
# https://circleci.com/docs/2.0/configuration-reference/#machine
# https://docs.docker.com/engine/release-notes/
machine:
image: ubuntu-1604:201903-01
docker_layer_caching: true
shell: bash
working_directory: ~/app
parameters:
docker-compose-up-service:
type: string
default: app
run-tests-csharp:
type: boolean
default: true
run-tests-spa-unit:
type: boolean
default: true
run-tests-spa-integration:
type: boolean
default: true
env-to-load:
type: string
default: default
build-args:
type: string
default: ""
# default is to build all of the things
build-target:
type: string
default: ""
docker-vols-root:
type: string
default: ~/docker-volumes
docker-image-repo:
type: string
default: quay.io
pre-build:
type: steps
description: "Steps to run prior to the build"
default: []
post-build:
type: steps
description: "Steps to run after the build"
default: []
push-images:
type: steps
description: "Steps to push the desired images to the configured repo. "
default:
- run:
name: "Publish the Release Docker image to the image repository"
command: docker push $RL_IMAGE
- run:
name: "Publish the Release Docker image (as latest) to the image repository"
command: docker tag $RL_IMAGE $RL_IMAGE_LATEST && docker push $RL_IMAGE_LATEST
steps:
- run:
name: "Login to the Docker image repository"
command: echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin << parameters.docker-image-repo >>
# Run the built-in checkout script to checkout the code in this repo and clone it in the machine vm
- checkout
- run:
name: "Prepare environment variables and docker container volumes"
command: |
echo "Clean up old docker volumes"
docker volume rm --force $(docker volume ls -q)
echo "Setting parameter-based environment variables..."
export RL_DOCKER_VOLS_ROOT=<< parameters.docker-vols-root >>
echo -e "\nSourcing..."
export EXEC_DIR="$(pwd)/.cicd/build"
source .cicd/build/load-env.sh -o $BASH_ENV << parameters.env-to-load >>
echo -e "\n\nBASH_ENV -> $BASH_ENV"
echo -e "\nContents -> $(cat $BASH_ENV)"
echo -e "\nVersions..."
echo -e "OpenSSL version -> $(openssl version)"
echo -e "Docker version -> $(docker --version)"
echo "Docker compose version -> $(docker-compose --version)"
echo -e "\n\n'docker-compose-test' function registered -> $(docker-compose-test version)"
source .cicd/build/make-docker-volumes.sh
ls -al ~
ls -alR $RL_DOCKER_VOLS_ROOT
- run:
name: "Make some certs"
command: source .cicd/build/make-local-certs.sh
# Download twistcli from the configured Twistlock console
# ...and ensure it's executable.
- run:
name: "Download twistcli in preparation for the Docker image security scan"
command: |
curl -k -u $TL_USER:$TL_PASS --output ~/app/twistcli $TL_CONSOLE_URL/api/v1/util/twistcli &&
sudo chmod a+x ./twistcli
- steps: << parameters.pre-build >>
- run:
name: "Build docker images (force pulling the latest base images)"
command: docker-compose-test build --pull << parameters.build-args >> << parameters.build-target >>
- steps: << parameters.post-build >>
- run:
name: "Bring up the images"
command: docker-compose-test pull --quiet --ignore-pull-failures && docker-compose-test up -d << parameters.docker-compose-up-service >>
- run:
name: "Document the runtime environment"
when: always
command: |
#todo(jhadlock): parameterize this?
echo -e "\nOur docker images..."
docker image ls --digests "<<REDACTED>>/*"
echo -e "\nVarious dependency docker images..."
docker image ls --digests "*"
echo -e "\nMicrosoft docker images..."
docker image ls --digests "microsoft/*"
echo -e "\ndocker-compose services running..."
docker-compose-test ps
- when:
condition: << parameters.run-tests-csharp >>
steps:
- run:
name: "Run csharp unit and integration tests..."
command: docker-compose-test run --no-deps tests-csharp
- when:
condition: << parameters.run-tests-spa-unit >>
steps:
- run:
name: "Run spa unit tests..."
command: docker-compose-test run --no-deps tests-spa-unit
- when:
condition: << parameters.run-tests-spa-integration >>
steps:
- run:
name: "Run spa integration tests..."
command: docker-compose-test run --no-deps tests-spa-integration
- store_artifacts:
name: "Uploading test artifacts..."
# pretty sure cannot use envvar subst here
path: << parameters.docker-vols-root >>/out
- sonarcloud/scan
# Run the scan with twistcli, providing detailed results in CircleCI and
# pushing the results to the Twistlock console.
# --details returns all vulnerabilities & compliance issues rather than just summaries.
# -address points to our Twistlock console
# -u and -p provide credentials for the console. These creds only need the CI User role.
# Finally, we provide the name of the image we built with 'docker build', above.
- run:
name: "Run the security scan over the Docker image"
command: |
./twistcli images scan $RL_IMAGE \
--details \
-address $TL_CONSOLE_URL \
-u $TL_USER \
-p $TL_PASS \
--vulnerability-threshold high \
--compliance-threshold high \
--only-fixed
# deploy image
- steps: << parameters.push-images >>
I know that’s a bit of a monster, but it gives a sense of the pipeline.
Greg